>>> kops: Building testing/kops 1.25.3-r1 (using abuild 3.10.0-r0) started Thu, 08 Dec 2022 11:23:34 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r4) (2/3) Installing go (1.19.4-r0) (3/3) Installing .makedepends-kops (20221208.112336) Executing busybox-1.35.0-r29.trigger OK: 801 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 31.1M 100 31.1M 0 0 41.0M 0 --:--:-- --:--:-- --:--:-- 41.0M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading k8s.io/api v0.25.4 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading google.golang.org/api v0.103.0 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading k8s.io/apimachinery v0.25.4 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading k8s.io/client-go v0.25.4 go: downloading go.uber.org/multierr v1.8.0 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading github.com/stretchr/testify v1.8.1 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/spf13/viper v1.14.0 go: downloading golang.org/x/crypto v0.3.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading go.uber.org/atomic v1.9.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/kubectl v0.25.4 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/google/go-cmp v0.5.9 go: downloading k8s.io/cloud-provider v0.25.4 go: downloading k8s.io/component-base v0.25.4 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/google/gnostic v0.6.9 go: downloading golang.org/x/net v0.2.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.2.0 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading golang.org/x/sys v0.2.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.1.0 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading google.golang.org/grpc v1.50.1 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.4.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading go.opencensus.io v0.24.0 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading github.com/onsi/gomega v1.20.2 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/containerd/containerd v1.6.6 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I1208 11:25:42.528047 30221 addon.go:233] addon test wants to update all nodes I1208 11:25:42.529727 30221 addon.go:233] addon test wants to update worker nodes I1208 11:25:42.530032 30221 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I1208 11:25:42.530464 30221 addon.go:266] installing PKI for "test" I1208 11:25:43.010938 30221 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (2.31s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 2.381s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.102s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I1208 11:25:57.595150 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.595358 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.595410 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.595456 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.595511 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.595592 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.595619 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.595644 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.595665 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.595741 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.595789 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.659375 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.659539 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.659878 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.669567 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.669683 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.669709 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.669732 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.669757 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.669777 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.669790 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.669807 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.669819 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.669865 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.669883 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.689025 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.689145 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.689354 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.693371 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.693470 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.693492 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.693514 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.693539 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.693557 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.693570 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.693587 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.693600 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.693642 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.693660 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.716365 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.716498 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.716695 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.720745 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.720844 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.720865 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.720886 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.720908 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.720926 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.720939 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.720955 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.720967 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.721010 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.721027 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.776832 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.776985 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.777252 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.784494 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.784629 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.784669 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.784693 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.784721 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.784754 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.784766 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.784783 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.784795 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.784841 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.784859 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.840569 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.840695 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.840970 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.845652 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.845772 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.845800 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.845822 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.845848 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.845869 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.845882 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.845901 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.845915 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.845960 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.845978 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.860125 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.860254 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.860470 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.864695 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.864812 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.864836 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.864858 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.864888 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.864908 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.864921 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.864939 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.864952 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.865016 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.865036 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.899284 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.899421 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.899652 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:57.907071 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.907281 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.907324 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.907364 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.907407 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.907443 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.907469 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.907502 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.907528 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.907605 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.907642 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.963859 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:57.963989 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:57.964389 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.38s) === RUN TestCreateClusterHetzner I1208 11:25:57.973314 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:57.973555 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:57.973604 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:57.973645 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:57.973687 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:57.973759 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:57.973785 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:57.973818 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:57.973842 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:57.973930 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:57.973970 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1208 11:25:58.031205 34645 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1208 11:25:58.031278 34645 new_cluster.go:1279] Cloud Provider ID = hetzner I1208 11:25:58.037262 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.037447 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.037494 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.037520 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.037550 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.037574 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.037601 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.037620 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.037633 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.037702 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.037737 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.189617 34645 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.23s) === RUN TestCreateClusterOverride I1208 11:25:58.198477 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.198695 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.198759 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.198814 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.198865 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.198941 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.198969 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.199005 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.199031 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.199111 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.199150 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.257665 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.258140 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.258414 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterComplex I1208 11:25:58.263413 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.263591 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.263625 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.263656 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.263688 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.263738 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.263754 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.263779 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.263796 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.263852 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.263874 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.296675 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.296839 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.297136 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I1208 11:25:58.301908 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.302098 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.302144 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.302172 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.302201 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.302251 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.302268 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.302289 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.302305 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.302365 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.302388 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.341408 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.341684 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.341696 34645 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1208 11:25:58.341705 34645 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1208 11:25:58.341987 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:58.350792 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.350981 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.351019 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.351063 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.351099 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.351128 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.351146 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.351172 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.351190 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.351257 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.351285 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.376387 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.376709 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.376727 34645 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1208 11:25:58.376737 34645 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1208 11:25:58.377044 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.08s) === RUN TestCreateClusterHAGCE I1208 11:25:58.385444 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.385606 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.385642 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.385667 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.385696 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.385748 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.385765 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.385786 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.385803 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.385861 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.385886 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.421389 34645 new_cluster.go:1279] Cloud Provider ID = gce I1208 11:25:58.421529 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.04s) === RUN TestCreateClusterGCE I1208 11:25:58.430260 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.430432 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.430467 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.430495 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.430525 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.430577 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.430594 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.430618 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.430635 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.430695 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.430722 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.519500 34645 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1208 11:25:58.519563 34645 new_cluster.go:1279] Cloud Provider ID = gce I1208 11:25:58.519634 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.09s) === RUN TestCreateClusterHASharedZone I1208 11:25:58.525286 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.525457 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.525492 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.525523 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.525554 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.525606 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.525626 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.525651 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.525669 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.525745 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.525774 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1208 11:25:58.579687 34645 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1208 11:25:58.579740 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.579964 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.580257 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.06s) === RUN TestCreateClusterHASharedZones I1208 11:25:58.587108 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.587271 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.587306 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.587332 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.587365 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.587417 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.587435 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.587458 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.587475 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.587533 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.587559 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1208 11:25:58.651313 34645 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1208 11:25:58.651406 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.652026 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.652051 34645 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1208 11:25:58.653173 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.08s) === RUN TestCreateClusterPrivate I1208 11:25:58.671336 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.671559 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.671605 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.671650 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.671695 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.671772 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.671803 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.671840 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.671867 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.671958 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.671999 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.781794 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.782016 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.782033 34645 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1208 11:25:58.782388 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:58.782439 34645 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.12s) === RUN TestCreateClusterPrivateGCE I1208 11:25:58.789041 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.789200 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.789239 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.789267 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.789298 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.789353 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.789371 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.789392 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.789409 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.789467 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.789494 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.815835 34645 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1208 11:25:58.815905 34645 new_cluster.go:1279] Cloud Provider ID = gce I1208 11:25:58.815991 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.04s) === RUN TestCreateClusterWithNGWSpecified I1208 11:25:58.827347 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.827580 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.827625 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.827667 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.827716 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.827796 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.827827 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.827866 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.827893 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.827974 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.828016 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.927533 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.927737 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.927754 34645 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1208 11:25:58.928058 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:58.928090 34645 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.11s) === RUN TestCreateClusterWithINGWSpecified I1208 11:25:58.934526 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.934669 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.934701 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.934727 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.934757 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.934810 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.934827 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.934847 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.934865 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.934922 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.934948 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.971143 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:58.971333 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:58.971350 34645 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1208 11:25:58.971618 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:58.971649 34645 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.04s) === RUN TestCreateClusterSharedVPC I1208 11:25:58.978207 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.978350 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.978383 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.978409 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.978439 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.978486 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.978504 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.978526 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.978542 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.978591 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:58.978617 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.025387 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.025562 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.025618 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.025656 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.025672 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.025715 34645 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I1208 11:25:59.026013 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.05s) === RUN TestCreateClusterSharedSubnets I1208 11:25:59.031607 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.031771 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.031803 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.031832 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.031862 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.031916 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.031934 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.031958 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.031976 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.032038 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.032062 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.087987 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.088037 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.088114 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.088259 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.088281 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.088306 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.088323 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.088647 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.06s) === RUN TestCreateClusterSharedSubnetsVpcLookup I1208 11:25:59.094350 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.094536 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.094568 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.094594 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.094627 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.094684 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.094702 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.094727 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.094744 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.094809 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.094838 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.126507 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:59.126583 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.126607 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.126673 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.126808 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.126828 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.126851 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.126865 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.127160 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I1208 11:25:59.132413 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.132565 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.132597 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.132622 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.132652 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.132704 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.132721 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.132742 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.132760 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.132838 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.132866 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.158119 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.158150 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.158204 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.158224 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.158249 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.158367 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.158387 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.158408 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:25:59.158421 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1208 11:25:59.158651 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.03s) === RUN TestCreateClusterIPv6 I1208 11:25:59.163841 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.163968 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.163996 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.164018 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.164042 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.164085 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.164102 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.164121 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.164135 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.164179 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.164203 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.201594 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.201764 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:59.202137 34645 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.04s) === RUN TestCreateClusterDifferentAMIs I1208 11:25:59.207893 34645 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1208 11:25:59.207907 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.208053 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.208086 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.208112 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.208144 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.208195 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.208213 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.208234 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.208253 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.208314 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.208339 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.227114 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.227200 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:59.227215 34645 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1208 11:25:59.227528 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.237319 34645 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.03s) === RUN TestCreateClusterKarpenter I1208 11:25:59.237416 34645 featureflag.go:160] FeatureFlag "Karpenter"=true I1208 11:25:59.237428 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.237605 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.237641 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.237672 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.237705 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.237761 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.237781 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.237805 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.237823 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.237886 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.237913 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1208 11:25:59.281284 34645 new_cluster.go:1279] Cloud Provider ID = aws I1208 11:25:59.281454 34645 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1208 11:25:59.281469 34645 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1208 11:25:59.281480 34645 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1208 11:25:59.281799 34645 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.287938 34645 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.05s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I1208 11:25:59.288174 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.288296 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.288323 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.288375 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.288404 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.288426 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.288443 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.288465 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.288480 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.293495 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I1208 11:25:59.294622 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.294756 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.294781 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.294803 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.294831 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.294873 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.294891 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.294909 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.294924 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.312135 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:25:59.314844 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:25:59.314939 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1208 11:26:02.326408 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:05.090594 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:05.091518 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:05.092352 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:05.092367 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:05.092420 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:05.092505 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:05.092639 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:05.094193 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:05.100857 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:05.101843 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:05.102376 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:05.102577 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:05.104779 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:05.104908 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:05.105393 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:05.106098 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:05.158445 34645 target.go:112] Terraform output is in /tmp/test554352685/out --- PASS: TestMinimal (5.87s) === RUN TestMinimal_v1_23 I1208 11:26:05.162676 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:05.162927 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:05.162987 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:05.163039 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:05.163091 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:05.163182 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:05.163212 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:05.163248 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:05.163278 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:05.203413 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1208 11:26:05.207586 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:05.207709 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.730568 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:06.866871 34645 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1208 11:26:06.867187 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:06.867291 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:06.867443 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:06.867476 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:06.867573 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:06.867595 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:06.869483 34645 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1208 11:26:06.872650 34645 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1208 11:26:06.872969 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.873075 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.875124 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.877794 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.877936 34645 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1208 11:26:06.878234 34645 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1208 11:26:06.878419 34645 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1208 11:26:06.917831 34645 target.go:112] Terraform output is in /tmp/test4006876624/out --- PASS: TestMinimal_v1_23 (1.76s) === RUN TestMinimal_v1_24 I1208 11:26:06.924427 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:06.924701 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:06.924759 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:06.924862 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:06.924917 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:06.924961 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:06.924990 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:06.925028 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:06.925057 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:06.951428 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:06.954786 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:06.954884 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.256491 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:08.441111 34645 executor.go:111] Tasks: 0 done / 93 total; 46 can run W1208 11:26:08.442463 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:08.443008 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:08.443052 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:08.443203 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:08.443307 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:08.443888 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:08.447963 34645 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1208 11:26:08.453946 34645 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1208 11:26:08.454569 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.454834 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.455802 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.458330 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.458457 34645 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1208 11:26:08.458835 34645 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1208 11:26:08.459050 34645 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1208 11:26:08.517622 34645 target.go:112] Terraform output is in /tmp/test3324614561/out --- PASS: TestMinimal_v1_24 (1.60s) === RUN TestMinimal_v1_25 I1208 11:26:08.522344 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:08.522612 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:08.522677 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:08.522730 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:08.522786 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:08.522882 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:08.522914 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:08.522952 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:08.522988 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:08.548930 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:08.552383 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:08.552489 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:09.916382 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:10.099827 34645 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1208 11:26:10.100382 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:10.100470 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:10.100631 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:10.100884 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:10.100916 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:10.101001 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:10.103266 34645 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1208 11:26:10.110554 34645 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1208 11:26:10.111018 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:10.111277 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:10.112494 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:10.113760 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:10.113883 34645 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1208 11:26:10.114187 34645 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1208 11:26:10.114445 34645 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1208 11:26:10.169181 34645 target.go:112] Terraform output is in /tmp/test2351935025/out --- PASS: TestMinimal_v1_25 (1.65s) === RUN TestMinimal_v1_26 I1208 11:26:10.173608 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:10.173855 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:10.173912 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:10.173963 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:10.174015 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:10.174105 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:10.174139 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:10.174176 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:10.174205 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:10.202745 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:10.206117 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:10.206223 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:11.329559 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:11.512149 34645 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1208 11:26:11.512503 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:11.512724 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:11.512760 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:11.512873 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:11.513224 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:11.513324 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:11.514885 34645 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1208 11:26:11.520109 34645 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1208 11:26:11.520331 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:11.520670 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:11.525856 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:11.527545 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:11.527680 34645 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1208 11:26:11.528027 34645 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1208 11:26:11.528196 34645 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1208 11:26:11.579440 34645 target.go:112] Terraform output is in /tmp/test510614582/out --- PASS: TestMinimal_v1_26 (1.41s) === RUN TestHetzner I1208 11:26:11.585930 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:12.210296 34645 executor.go:111] Tasks: 0 done / 50 total; 39 can run I1208 11:26:12.216878 34645 executor.go:111] Tasks: 39 done / 50 total; 5 can run I1208 11:26:12.218256 34645 executor.go:111] Tasks: 44 done / 50 total; 2 can run I1208 11:26:12.224171 34645 executor.go:111] Tasks: 46 done / 50 total; 4 can run I1208 11:26:12.226826 34645 executor.go:111] Tasks: 50 done / 50 total; 0 can run I1208 11:26:12.245726 34645 target.go:112] Terraform output is in /tmp/test1459054617/out --- PASS: TestHetzner (0.66s) === RUN TestNvidia I1208 11:26:12.249375 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:12.249745 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:12.249840 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:12.250045 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:12.250102 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:12.250143 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:12.250172 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:12.250210 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:12.250238 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:12.275265 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:12.278343 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.278440 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.319760 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:12.431505 34645 executor.go:111] Tasks: 0 done / 88 total; 44 can run I1208 11:26:12.431891 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:12.431924 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:12.432016 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:12.432285 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:12.432406 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:12.432479 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:12.434227 34645 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1208 11:26:12.439601 34645 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1208 11:26:12.439846 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.440030 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.441267 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.442304 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.442429 34645 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1208 11:26:12.442757 34645 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1208 11:26:12.443018 34645 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1208 11:26:12.499534 34645 target.go:112] Terraform output is in /tmp/test3842428605/out --- PASS: TestNvidia (0.25s) === RUN TestMinimalGossip I1208 11:26:12.503992 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:12.504249 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:12.504309 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:12.504367 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:12.504420 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:12.504513 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:12.504543 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:12.504581 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:12.504612 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:12.531382 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.531475 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.672690 34645 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1208 11:26:12.674011 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:12.674484 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:12.674646 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:12.679411 34645 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1208 11:26:12.686013 34645 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1208 11:26:12.686400 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.686775 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.688046 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.689411 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.689530 34645 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1208 11:26:12.689844 34645 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1208 11:26:12.690105 34645 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1208 11:26:12.741267 34645 target.go:112] Terraform output is in /tmp/test1943569630/out --- PASS: TestMinimalGossip (0.24s) === RUN TestMinimalGossipIRSA I1208 11:26:12.745213 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:12.745477 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:12.745535 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:12.745584 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:12.745639 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:12.745730 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:12.745761 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:12.745798 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:12.745830 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:12.776996 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.777100 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.924328 34645 executor.go:111] Tasks: 0 done / 89 total; 44 can run I1208 11:26:12.924767 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:12.925001 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:12.925008 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:12.926810 34645 executor.go:111] Tasks: 44 done / 89 total; 20 can run I1208 11:26:12.932238 34645 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1208 11:26:12.932604 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.932814 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.933708 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.935084 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:12.935200 34645 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1208 11:26:12.935528 34645 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1208 11:26:12.935782 34645 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1208 11:26:12.988298 34645 target.go:112] Terraform output is in /tmp/test3053148897/out --- PASS: TestMinimalGossipIRSA (0.25s) === RUN TestMinimalGCE I1208 11:26:12.992620 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:12.994527 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1208 11:26:13.885727 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:13.885896 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:13.886157 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1208 11:26:13.886226 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1208 11:26:13.886265 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:13.886594 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:13.886965 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:13.889534 34645 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1208 11:26:13.892728 34645 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1208 11:26:13.898181 34645 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1208 11:26:13.900943 34645 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1208 11:26:13.901475 34645 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1208 11:26:13.931425 34645 target.go:112] Terraform output is in /tmp/test706956905/out --- PASS: TestMinimalGCE (0.94s) === RUN TestMinimalGCEPrivate I1208 11:26:13.935328 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:13.937151 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:14.564773 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:14.564870 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:14.565002 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W1208 11:26:14.565040 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W1208 11:26:14.565066 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1208 11:26:14.565102 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1208 11:26:14.565175 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:14.567049 34645 executor.go:111] Tasks: 0 done / 66 total; 40 can run I1208 11:26:14.569647 34645 executor.go:111] Tasks: 40 done / 66 total; 18 can run I1208 11:26:14.578352 34645 executor.go:111] Tasks: 58 done / 66 total; 4 can run I1208 11:26:14.582601 34645 executor.go:111] Tasks: 62 done / 66 total; 3 can run I1208 11:26:14.584103 34645 executor.go:111] Tasks: 65 done / 66 total; 1 can run I1208 11:26:14.584377 34645 executor.go:111] Tasks: 66 done / 66 total; 0 can run I1208 11:26:14.619449 34645 target.go:112] Terraform output is in /tmp/test3164769135/out --- PASS: TestMinimalGCEPrivate (0.69s) === RUN TestMinimalGCEInternalLoadBalancer I1208 11:26:14.623120 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:14.625056 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:14.808194 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:14.808267 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:14.808400 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W1208 11:26:14.808436 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W1208 11:26:14.808463 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1208 11:26:14.808497 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1208 11:26:14.808560 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1208 11:26:14.808604 34645 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1208 11:26:14.810462 34645 executor.go:111] Tasks: 0 done / 68 total; 41 can run I1208 11:26:14.813965 34645 executor.go:111] Tasks: 41 done / 68 total; 17 can run W1208 11:26:14.819023 34645 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1208 11:26:14.819060 34645 executor.go:111] Tasks: 58 done / 68 total; 4 can run I1208 11:26:14.821565 34645 executor.go:111] Tasks: 62 done / 68 total; 3 can run I1208 11:26:14.822856 34645 executor.go:111] Tasks: 65 done / 68 total; 2 can run I1208 11:26:14.823160 34645 executor.go:111] Tasks: 67 done / 68 total; 1 can run I1208 11:26:14.823411 34645 executor.go:111] Tasks: 68 done / 68 total; 0 can run I1208 11:26:14.857363 34645 target.go:112] Terraform output is in /tmp/test2806358268/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.24s) === RUN TestMinimalGCELongClusterName I1208 11:26:14.861341 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:14.863297 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1208 11:26:15.099317 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:15.099429 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:15.099570 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1208 11:26:15.099610 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1208 11:26:15.099639 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1208 11:26:15.099675 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1208 11:26:15.099759 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:15.101458 34645 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1208 11:26:15.104253 34645 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1208 11:26:15.109846 34645 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1208 11:26:15.112514 34645 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1208 11:26:15.112776 34645 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1208 11:26:15.144405 34645 target.go:112] Terraform output is in /tmp/test4142149771/out --- PASS: TestMinimalGCELongClusterName (0.29s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I1208 11:26:15.148640 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:15.150654 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1208 11:26:15.389972 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:15.390051 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:15.390181 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1208 11:26:15.390213 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1208 11:26:15.390235 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1208 11:26:15.390265 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1208 11:26:15.390323 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1208 11:26:15.390368 34645 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1208 11:26:15.391999 34645 executor.go:111] Tasks: 0 done / 67 total; 42 can run I1208 11:26:15.398917 34645 executor.go:111] Tasks: 42 done / 67 total; 16 can run W1208 11:26:15.404782 34645 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1208 11:26:15.404820 34645 executor.go:111] Tasks: 58 done / 67 total; 5 can run I1208 11:26:15.407908 34645 executor.go:111] Tasks: 63 done / 67 total; 2 can run I1208 11:26:15.408199 34645 executor.go:111] Tasks: 65 done / 67 total; 1 can run I1208 11:26:15.408378 34645 executor.go:111] Tasks: 66 done / 67 total; 1 can run I1208 11:26:15.408552 34645 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1208 11:26:15.440868 34645 target.go:112] Terraform output is in /tmp/test2191087004/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.30s) === RUN TestHA I1208 11:26:15.444828 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:15.445091 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:15.445148 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:15.445203 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:15.445258 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:15.445356 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:15.445391 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:15.445456 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:15.445486 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:15.477693 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:15.480772 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.480856 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.480906 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.480952 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.521273 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:15.639853 34645 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1208 11:26:15.641110 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:15.641546 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:15.641585 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:15.641547 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:15.645347 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:15.645392 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:15.645387 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:15.645534 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:15.647565 34645 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1208 11:26:15.653432 34645 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1208 11:26:15.653741 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.654775 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.655529 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.655749 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.659670 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.660397 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.660879 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.660963 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:15.661104 34645 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1208 11:26:15.661446 34645 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1208 11:26:15.661774 34645 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1208 11:26:15.739737 34645 target.go:112] Terraform output is in /tmp/test3873693862/out --- PASS: TestHA (0.30s) === RUN TestHighAvailabilityGCE I1208 11:26:15.744736 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:15.746909 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:15.940858 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:15.940963 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:15.941088 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W1208 11:26:15.941126 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W1208 11:26:15.941151 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1208 11:26:15.941185 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1208 11:26:15.941255 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1208 11:26:15.941334 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1208 11:26:15.941402 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:15.943817 34645 executor.go:111] Tasks: 0 done / 83 total; 48 can run I1208 11:26:15.946670 34645 executor.go:111] Tasks: 48 done / 83 total; 20 can run I1208 11:26:15.957433 34645 executor.go:111] Tasks: 68 done / 83 total; 7 can run I1208 11:26:15.961121 34645 executor.go:111] Tasks: 75 done / 83 total; 5 can run I1208 11:26:15.962385 34645 executor.go:111] Tasks: 80 done / 83 total; 3 can run I1208 11:26:15.962680 34645 executor.go:111] Tasks: 83 done / 83 total; 0 can run I1208 11:26:16.012833 34645 target.go:112] Terraform output is in /tmp/test2062790281/out --- PASS: TestHighAvailabilityGCE (0.27s) === RUN TestComplex I1208 11:26:16.017749 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:16.017996 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:16.018056 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:16.018168 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:16.018222 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:16.018268 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:16.018300 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:16.018339 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:16.018369 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:16.050604 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:16.054903 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:16.055011 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:16.972831 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:17.169000 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1208 11:26:17.169347 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:17.169933 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:17.173934 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:17.173985 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:17.174147 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:17.175948 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:17.182343 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:17.183135 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.183436 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.184736 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.186105 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.186229 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:17.187709 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:17.188144 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1208 11:26:17.261653 34645 target.go:112] Terraform output is in /tmp/test2952577080/out I1208 11:26:17.268516 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:17.268803 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:17.268866 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:17.268919 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:17.268974 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:17.269016 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:17.269045 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:17.269081 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:17.269110 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:17.302119 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:17.305859 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.305946 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.346787 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:17.548285 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1208 11:26:17.549145 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:17.549388 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:17.549390 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:17.549439 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:17.549587 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:17.551130 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:17.559734 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:17.560458 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.560742 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.562454 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.563764 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.563884 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:17.564294 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:17.564511 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1208 11:26:17.637225 34645 target.go:112] Terraform output is in /tmp/test2566765031/out --- PASS: TestComplex (1.62s) === RUN TestCompress I1208 11:26:17.642028 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:17.642295 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:17.642355 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:17.642407 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:17.642461 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:17.642551 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:17.642582 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:17.642619 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:17.642648 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:17.670448 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:17.673584 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.673689 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.713704 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:17.817861 34645 executor.go:111] Tasks: 0 done / 86 total; 42 can run I1208 11:26:17.818363 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:17.818435 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:17.818392 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:17.818504 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:17.818937 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:17.821133 34645 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1208 11:26:17.826538 34645 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1208 11:26:17.826863 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.827279 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.833406 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.834892 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.836116 34645 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1208 11:26:17.836473 34645 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1208 11:26:17.836708 34645 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1208 11:26:17.881042 34645 target.go:112] Terraform output is in /tmp/test4261896745/out --- PASS: TestCompress (0.25s) === RUN TestExternalPolicies I1208 11:26:17.887456 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:17.887880 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:17.887983 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:17.888156 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:17.888261 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:17.888337 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:17.888392 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:17.888437 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:17.888469 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:17.916688 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:17.920136 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.920230 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:17.959859 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:18.058508 34645 executor.go:111] Tasks: 0 done / 100 total; 45 can run W1208 11:26:18.058958 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:18.059140 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:18.059168 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:18.059192 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:18.059280 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:18.059325 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:18.060989 34645 executor.go:111] Tasks: 45 done / 100 total; 20 can run I1208 11:26:18.065217 34645 executor.go:111] Tasks: 65 done / 100 total; 30 can run I1208 11:26:18.065737 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.065891 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.066853 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.068243 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.068360 34645 executor.go:111] Tasks: 95 done / 100 total; 3 can run I1208 11:26:18.068802 34645 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1208 11:26:18.069025 34645 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1208 11:26:18.136270 34645 target.go:112] Terraform output is in /tmp/test3580609392/out --- PASS: TestExternalPolicies (0.25s) === RUN TestMinimalIPv6 I1208 11:26:18.140520 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:18.140813 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:18.140873 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:18.140934 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:18.140999 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:18.141105 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:18.141137 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:18.141189 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:18.141222 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:18.166700 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:18.170332 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.170431 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.210266 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:18.378778 34645 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1208 11:26:18.379393 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:18.379454 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:18.379598 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:18.379645 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:18.379790 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:18.379846 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:18.381854 34645 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1208 11:26:18.387845 34645 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1208 11:26:18.388358 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.388579 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.393684 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.394728 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.394906 34645 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1208 11:26:18.395512 34645 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1208 11:26:18.395858 34645 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1208 11:26:18.459849 34645 target.go:112] Terraform output is in /tmp/test543109446/out --- PASS: TestMinimalIPv6 (0.32s) === RUN TestMinimalIPv6Private I1208 11:26:18.464084 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:18.464327 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:18.464382 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:18.464430 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:18.464484 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:18.464572 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:18.464601 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:18.464640 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:18.464669 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:18.493915 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1208 11:26:18.497626 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.497713 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.538287 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:18.717313 34645 executor.go:111] Tasks: 0 done / 124 total; 46 can run I1208 11:26:18.717882 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:18.717984 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:18.717942 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:18.718086 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:18.718228 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:18.718314 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:18.720772 34645 executor.go:111] Tasks: 46 done / 124 total; 23 can run I1208 11:26:18.731386 34645 executor.go:111] Tasks: 69 done / 124 total; 36 can run I1208 11:26:18.731948 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.732180 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.733231 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.734788 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.734921 34645 executor.go:111] Tasks: 105 done / 124 total; 11 can run I1208 11:26:18.735469 34645 executor.go:111] Tasks: 116 done / 124 total; 8 can run I1208 11:26:18.735929 34645 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1208 11:26:18.811573 34645 target.go:112] Terraform output is in /tmp/test1226202056/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I1208 11:26:18.816181 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:18.816465 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:18.816526 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:18.816638 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:18.816695 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:18.816740 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:18.816791 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:18.816830 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:18.816862 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:18.846053 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:18.850618 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.850729 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:18.891213 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:19.364434 34645 executor.go:111] Tasks: 0 done / 109 total; 47 can run I1208 11:26:19.364910 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:19.364980 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:19.365193 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:19.365333 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:19.365160 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:19.365521 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:19.367807 34645 executor.go:111] Tasks: 47 done / 109 total; 21 can run I1208 11:26:19.373480 34645 executor.go:111] Tasks: 68 done / 109 total; 31 can run I1208 11:26:19.373900 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.374119 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.375293 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.376610 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.376742 34645 executor.go:111] Tasks: 99 done / 109 total; 5 can run I1208 11:26:19.377238 34645 executor.go:111] Tasks: 104 done / 109 total; 5 can run I1208 11:26:19.377619 34645 executor.go:111] Tasks: 109 done / 109 total; 0 can run I1208 11:26:19.441051 34645 target.go:112] Terraform output is in /tmp/test2707734940/out --- PASS: TestMinimalIPv6Calico (0.63s) === RUN TestMinimalIPv6Cilium I1208 11:26:19.446688 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:19.446941 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:19.447002 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:19.447055 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:19.447108 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:19.447200 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:19.447234 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:19.447273 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:19.447303 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:19.477448 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:19.481179 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.481278 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.520491 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:19.735548 34645 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1208 11:26:19.737050 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:19.737901 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:19.738246 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:19.738489 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:19.738533 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:19.738685 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:19.742961 34645 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1208 11:26:19.748235 34645 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1208 11:26:19.748477 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.748884 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.750136 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.751808 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.751946 34645 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1208 11:26:19.752663 34645 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1208 11:26:19.753094 34645 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1208 11:26:19.818893 34645 target.go:112] Terraform output is in /tmp/test3666502211/out --- PASS: TestMinimalIPv6Cilium (0.38s) === RUN TestMinimalWarmPool I1208 11:26:19.823395 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:19.823640 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:19.823698 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:19.823804 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:19.823860 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:19.823907 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:19.823939 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:19.823975 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:19.824006 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:19.849368 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:19.853096 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.853196 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:19.892276 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:20.106560 34645 executor.go:111] Tasks: 0 done / 89 total; 45 can run I1208 11:26:20.106978 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:20.107301 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:20.107636 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.107637 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:20.107697 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:20.107827 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:20.110101 34645 executor.go:111] Tasks: 45 done / 89 total; 19 can run I1208 11:26:20.119744 34645 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1208 11:26:20.119946 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.120295 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.121136 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.123165 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.123377 34645 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1208 11:26:20.123900 34645 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1208 11:26:20.124122 34645 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1208 11:26:20.179838 34645 target.go:112] Terraform output is in /tmp/test4032512085/out --- PASS: TestMinimalWarmPool (0.36s) === RUN TestMinimalEtcd I1208 11:26:20.184456 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:20.184711 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:20.184765 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:20.184821 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:20.184874 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:20.184964 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:20.184997 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:20.185036 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:20.185065 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:20.211634 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:20.214891 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.214983 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.255846 34645 zones.go:93] ListHostedZonesPages { } W1208 11:26:20.343976 34645 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W1208 11:26:20.344039 34645 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W1208 11:26:20.346236 34645 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I1208 11:26:20.359738 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:20.360285 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:20.360532 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:20.360631 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.360645 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:20.360685 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:20.360823 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:20.363405 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:20.370664 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:20.371224 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.371502 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.372664 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.374008 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.374128 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:20.374455 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:20.374674 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:20.429255 34645 target.go:112] Terraform output is in /tmp/test3301979019/out --- PASS: TestMinimalEtcd (0.25s) === RUN TestMinimalGp3 I1208 11:26:20.433299 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:20.433578 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:20.433653 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:20.433731 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:20.433822 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:20.433950 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:20.434015 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:20.434087 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:20.434133 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:20.459328 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:20.462586 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.462694 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.502318 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:20.610098 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:20.610377 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:20.610706 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.611041 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:20.611055 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.611094 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:20.611237 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:20.612921 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:20.618170 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:20.618535 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.618675 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.619792 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.620878 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.620995 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:20.621296 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:20.621496 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:20.672628 34645 target.go:112] Terraform output is in /tmp/test3572528732/out --- PASS: TestMinimalGp3 (0.24s) === RUN TestMinimalLongClusterName I1208 11:26:20.676600 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:20.676880 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:20.676936 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:20.676994 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:20.677048 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:20.677146 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:20.677177 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:20.677215 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:20.677244 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:20.701191 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:20.704356 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.704454 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1208 11:26:20.742897 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:20.856542 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1208 11:26:20.857184 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.857254 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:20.857439 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:20.857479 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:20.857623 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:20.857634 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:20.860225 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:20.867072 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:20.867765 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.867935 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.871665 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.872895 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.873032 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:20.873369 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:20.873599 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:20.930438 34645 target.go:112] Terraform output is in /tmp/test618495109/out --- PASS: TestMinimalLongClusterName (0.26s) === RUN TestExistingSG I1208 11:26:20.935095 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:20.935358 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:20.935438 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:20.935496 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:20.935551 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:20.935649 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:20.935680 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:20.935717 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:20.935750 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:20.969160 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:20.972994 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.973092 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.973152 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:20.973210 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.013583 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:21.129389 34645 executor.go:111] Tasks: 0 done / 146 total; 53 can run I1208 11:26:21.129953 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:21.130029 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:21.130217 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.130478 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:21.130604 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.130527 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:21.130655 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:21.130661 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.132856 34645 executor.go:111] Tasks: 53 done / 146 total; 26 can run I1208 11:26:21.144621 34645 executor.go:111] Tasks: 79 done / 146 total; 58 can run I1208 11:26:21.144852 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.145152 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.145428 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.145661 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.146723 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.147789 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.148226 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.148492 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.148624 34645 executor.go:111] Tasks: 137 done / 146 total; 5 can run I1208 11:26:21.149150 34645 executor.go:111] Tasks: 142 done / 146 total; 4 can run I1208 11:26:21.149393 34645 executor.go:111] Tasks: 146 done / 146 total; 0 can run I1208 11:26:21.247758 34645 target.go:112] Terraform output is in /tmp/test2888238438/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I1208 11:26:21.253147 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:21.253414 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:21.253477 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:21.253535 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:21.253590 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:21.253689 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:21.253721 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:21.253759 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:21.253788 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:21.283319 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:21.286540 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.286629 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.286688 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.325652 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:21.443891 34645 executor.go:111] Tasks: 0 done / 122 total; 48 can run I1208 11:26:21.444577 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:21.444757 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:21.444931 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.445162 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:21.445167 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.445207 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:21.445337 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:21.446825 34645 executor.go:111] Tasks: 48 done / 122 total; 29 can run I1208 11:26:21.455350 34645 executor.go:111] Tasks: 77 done / 122 total; 36 can run I1208 11:26:21.455685 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.456010 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.456230 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.457433 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.457922 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.458399 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.458522 34645 executor.go:111] Tasks: 113 done / 122 total; 5 can run I1208 11:26:21.459027 34645 executor.go:111] Tasks: 118 done / 122 total; 4 can run I1208 11:26:21.459344 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1208 11:26:21.540477 34645 target.go:112] Terraform output is in /tmp/test1245774411/out --- PASS: TestBastionAdditionalUserData (0.29s) === RUN TestPrivateWeave I1208 11:26:21.545231 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:21.545509 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:21.545570 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:21.545623 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:21.545678 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:21.545775 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:21.545806 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:21.545845 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:21.545877 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:21.579335 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:21.582656 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.582761 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.582823 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.625099 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:21.755050 34645 executor.go:111] Tasks: 0 done / 120 total; 47 can run I1208 11:26:21.755817 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:21.755873 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.755863 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:21.755981 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:21.755982 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:21.756031 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:21.756024 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:21.757911 34645 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1208 11:26:21.767702 34645 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1208 11:26:21.768253 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.768445 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.768528 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.768773 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.769897 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.770882 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.771016 34645 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1208 11:26:21.771537 34645 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1208 11:26:21.771935 34645 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1208 11:26:21.852737 34645 target.go:112] Terraform output is in /tmp/test2646130368/out --- PASS: TestPrivateWeave (0.31s) === RUN TestPrivateFlannel I1208 11:26:21.857131 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:21.857376 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:21.857435 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:21.857541 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:21.857592 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:21.857637 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:21.857668 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:21.857706 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:21.857737 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:21.885581 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:21.889061 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.889152 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.889212 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:21.927623 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:22.131421 34645 executor.go:111] Tasks: 0 done / 123 total; 50 can run I1208 11:26:22.132009 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:22.132051 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:22.132170 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:22.132195 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:22.132345 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:22.132445 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:22.132584 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:22.134384 34645 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1208 11:26:22.139811 34645 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1208 11:26:22.140291 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.140540 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.140634 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.140910 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.141950 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.143473 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.143598 34645 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1208 11:26:22.144061 34645 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1208 11:26:22.144324 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:22.223724 34645 target.go:112] Terraform output is in /tmp/test3121780414/out --- PASS: TestPrivateFlannel (0.37s) === RUN TestPrivateCalico I1208 11:26:22.228550 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:22.228821 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:22.228884 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:22.228992 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:22.229044 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:22.229087 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:22.229118 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:22.229155 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:22.229185 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:22.255775 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:22.258543 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.258623 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.258676 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.299352 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:22.778113 34645 executor.go:111] Tasks: 0 done / 127 total; 50 can run W1208 11:26:22.778786 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:22.779089 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:22.779091 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:22.779165 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:22.779374 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:22.779419 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:22.779556 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:22.784205 34645 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1208 11:26:22.790778 34645 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1208 11:26:22.791333 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.791478 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.791707 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.791833 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.793049 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.794279 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.794404 34645 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1208 11:26:22.794912 34645 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1208 11:26:22.795231 34645 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1208 11:26:22.876725 34645 target.go:112] Terraform output is in /tmp/test1220099239/out --- PASS: TestPrivateCalico (0.65s) === RUN TestPrivateCilium I1208 11:26:22.883555 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:22.884107 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:22.884456 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:22.884554 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:22.884617 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:22.884774 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:22.884807 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:22.884849 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:22.884881 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:22.914913 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:22.918573 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.918673 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.918754 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:22.959902 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:23.112569 34645 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1208 11:26:23.114194 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:23.114739 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:23.114783 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:23.114910 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:23.115603 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:23.113368 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:23.116287 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:23.119877 34645 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1208 11:26:23.125584 34645 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1208 11:26:23.125981 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.126289 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.126520 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.126701 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.127451 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.128564 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.128719 34645 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1208 11:26:23.129206 34645 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1208 11:26:23.129515 34645 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1208 11:26:23.207049 34645 target.go:112] Terraform output is in /tmp/test198355467/out --- PASS: TestPrivateCilium (0.33s) === RUN TestPrivateCilium2 I1208 11:26:23.211404 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:23.211669 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:23.211728 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:23.211834 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:23.211891 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:23.211933 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:23.211961 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:23.211998 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:23.212026 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:23.244657 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:23.248934 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.249142 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.249217 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.288152 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:23.983119 34645 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1208 11:26:23.983885 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:23.983990 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:23.984179 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:23.984272 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:23.984437 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:23.984477 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:23.984625 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:23.987225 34645 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1208 11:26:23.997496 34645 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1208 11:26:23.997702 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.998151 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.998305 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.998562 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:23.999008 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.000994 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.001123 34645 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1208 11:26:24.001539 34645 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1208 11:26:24.001839 34645 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1208 11:26:24.076773 34645 target.go:112] Terraform output is in /tmp/test2440169127/out --- PASS: TestPrivateCilium2 (0.87s) === RUN TestPrivateCiliumAdvanced I1208 11:26:24.083900 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:24.084151 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:24.084211 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:24.084263 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:24.084318 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:24.084408 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:24.084437 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:24.084477 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:24.084505 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:24.122362 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:24.126557 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.126658 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.126725 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.166084 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:24.327996 34645 executor.go:111] Tasks: 0 done / 126 total; 53 can run I1208 11:26:24.328484 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:24.328509 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:24.328590 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:24.329547 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:24.330041 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:24.330347 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:24.332391 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:24.336429 34645 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1208 11:26:24.342476 34645 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1208 11:26:24.342888 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.343079 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.343361 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.343486 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.345060 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.346062 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.346201 34645 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1208 11:26:24.346671 34645 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1208 11:26:24.347019 34645 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1208 11:26:24.427959 34645 target.go:112] Terraform output is in /tmp/test3476240230/out --- PASS: TestPrivateCiliumAdvanced (0.35s) === RUN TestPrivateCanal I1208 11:26:24.432724 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:24.432980 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:24.433046 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:24.433156 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:24.433211 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:24.433257 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:24.433285 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:24.433325 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:24.433353 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:24.466059 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:24.469532 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.469638 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.469702 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.508827 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:24.990613 34645 executor.go:111] Tasks: 0 done / 123 total; 50 can run I1208 11:26:24.990938 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:24.990984 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:24.991134 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:24.991444 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:24.991522 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:24.991650 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:24.991678 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:24.993489 34645 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1208 11:26:24.998926 34645 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1208 11:26:24.999455 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.999604 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:24.999847 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:25.000044 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:25.000851 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:25.001993 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:25.002138 34645 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1208 11:26:25.002593 34645 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1208 11:26:25.002907 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:25.081815 34645 target.go:112] Terraform output is in /tmp/test1968276949/out --- PASS: TestPrivateCanal (0.65s) === RUN TestPrivateKopeio I1208 11:26:25.087330 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:25.087576 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:25.087631 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:25.087680 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:25.087733 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:25.087820 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:25.087850 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:25.087891 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:25.087921 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:25.117920 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:25.121208 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.121300 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.121362 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.159686 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:25.278884 34645 executor.go:111] Tasks: 0 done / 126 total; 47 can run W1208 11:26:25.279559 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.279805 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:25.279812 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:25.279930 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:25.281393 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.279858 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:25.281976 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:25.286887 34645 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1208 11:26:25.292815 34645 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1208 11:26:25.293237 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.293441 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.293589 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.293784 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.294883 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.296187 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.296379 34645 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1208 11:26:25.297015 34645 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1208 11:26:25.297285 34645 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1208 11:26:25.379737 34645 target.go:112] Terraform output is in /tmp/test2402066008/out --- PASS: TestPrivateKopeio (0.30s) === RUN TestUnmanaged I1208 11:26:25.384349 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:25.384613 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:25.384673 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:25.384728 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:25.384784 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:25.384897 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:25.384931 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:25.384968 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:25.384997 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:25.415207 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:25.418311 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.418395 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.418451 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.458204 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:25.569118 34645 executor.go:111] Tasks: 0 done / 108 total; 45 can run W1208 11:26:25.569472 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.569841 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:25.569865 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:25.572648 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.572844 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:25.569887 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:25.573290 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:25.577652 34645 executor.go:111] Tasks: 45 done / 108 total; 25 can run I1208 11:26:25.582827 34645 executor.go:111] Tasks: 70 done / 108 total; 31 can run I1208 11:26:25.583387 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.583574 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.583865 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.584060 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.584724 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.585944 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.586094 34645 executor.go:111] Tasks: 101 done / 108 total; 4 can run I1208 11:26:25.586606 34645 executor.go:111] Tasks: 105 done / 108 total; 3 can run I1208 11:26:25.587029 34645 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1208 11:26:25.660492 34645 target.go:112] Terraform output is in /tmp/test3531499388/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I1208 11:26:25.664759 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:25.665026 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:25.665088 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:25.665141 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:25.665192 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:25.665290 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:25.665321 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:25.665361 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:25.665393 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:25.697443 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:25.701495 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.701664 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.701775 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.741502 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:25.855329 34645 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1208 11:26:25.855811 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:25.855934 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.856378 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:25.856415 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:25.856433 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:25.856422 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:25.856579 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:25.858485 34645 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1208 11:26:25.859067 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:25.864039 34645 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1208 11:26:25.864635 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.864848 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.865147 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.866387 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.867173 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.871562 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.871728 34645 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1208 11:26:25.872229 34645 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1208 11:26:25.872512 34645 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1208 11:26:25.941601 34645 target.go:112] Terraform output is in /tmp/test1306432521/out --- PASS: TestPrivateSharedSubnet (0.28s) === RUN TestPrivateSharedIP I1208 11:26:25.945950 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:25.946211 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:25.946274 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:25.946327 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:25.946379 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:25.946479 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:25.946509 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:25.946546 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:25.946575 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:25.975511 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:25.978913 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.979004 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:25.979065 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.018559 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:26.127864 34645 executor.go:111] Tasks: 0 done / 116 total; 45 can run W1208 11:26:26.128113 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:26.128222 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:26.128377 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:26.128350 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:26.128695 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:26.128746 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:26.128847 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:26.130525 34645 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1208 11:26:26.130983 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:26.139578 34645 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1208 11:26:26.139933 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.140249 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.140404 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.140693 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.141684 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.142792 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:26.142916 34645 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1208 11:26:26.143485 34645 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1208 11:26:26.143745 34645 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1208 11:26:26.217308 34645 target.go:112] Terraform output is in /tmp/test4220710357/out --- PASS: TestPrivateSharedIP (0.28s) === RUN TestPrivateDns1 I1208 11:26:26.221479 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:26.221734 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:26.221794 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:26.221845 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:26.221900 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:26.221995 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:26.222026 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:26.222081 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:26.222115 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:26.259305 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.259405 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.259468 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.417729 34645 executor.go:111] Tasks: 0 done / 120 total; 46 can run W1208 11:26:26.418427 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:26.418446 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:26.418515 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:26.418567 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:26.421305 34645 executor.go:111] Tasks: 46 done / 120 total; 24 can run I1208 11:26:26.421876 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I1208 11:26:26.421918 34645 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I1208 11:26:26.422012 34645 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I1208 11:26:26.422029 34645 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I1208 11:26:26.422124 34645 executor.go:111] Tasks: 70 done / 120 total; 35 can run I1208 11:26:26.422530 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.423952 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.429925 34645 executor.go:111] Tasks: 105 done / 120 total; 5 can run W1208 11:26:26.430005 34645 api.go:53] PageSize not implemented W1208 11:26:26.430107 34645 api.go:53] PageSize not implemented I1208 11:26:26.435239 34645 executor.go:111] Tasks: 110 done / 120 total; 6 can run I1208 11:26:26.435507 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.435782 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.437754 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.439209 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.439356 34645 executor.go:111] Tasks: 116 done / 120 total; 2 can run I1208 11:26:26.439753 34645 executor.go:111] Tasks: 118 done / 120 total; 2 can run I1208 11:26:26.440053 34645 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1208 11:26:26.526068 34645 target.go:112] Terraform output is in /tmp/test1131530347/out --- PASS: TestPrivateDns1 (0.31s) === RUN TestPrivateDns2 I1208 11:26:26.530548 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:26.530822 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:26.530896 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:26.530975 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:26.531067 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:26.531182 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:26.531241 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:26.531308 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:26.531360 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:26.563680 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.563780 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.563848 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.700132 34645 executor.go:111] Tasks: 0 done / 116 total; 44 can run W1208 11:26:26.700772 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:26.700790 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:26.701054 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:26.701367 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:26.703560 34645 executor.go:111] Tasks: 44 done / 116 total; 22 can run I1208 11:26:26.703942 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:26.704064 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I1208 11:26:26.704100 34645 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I1208 11:26:26.704181 34645 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I1208 11:26:26.704201 34645 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I1208 11:26:26.704266 34645 executor.go:111] Tasks: 66 done / 116 total; 35 can run I1208 11:26:26.704953 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.705188 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.707872 34645 executor.go:111] Tasks: 101 done / 116 total; 5 can run W1208 11:26:26.708049 34645 api.go:53] PageSize not implemented W1208 11:26:26.708134 34645 api.go:53] PageSize not implemented I1208 11:26:26.716774 34645 executor.go:111] Tasks: 106 done / 116 total; 6 can run I1208 11:26:26.717064 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.717187 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.718360 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.719506 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.719628 34645 executor.go:111] Tasks: 112 done / 116 total; 2 can run I1208 11:26:26.719945 34645 executor.go:111] Tasks: 114 done / 116 total; 2 can run I1208 11:26:26.720147 34645 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1208 11:26:26.793301 34645 target.go:112] Terraform output is in /tmp/test824699856/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I1208 11:26:26.797659 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:26.797917 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:26.797978 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:26.798083 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:26.798139 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:26.798187 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:26.798217 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:26.798258 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:26.798287 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:26.824184 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:26.827672 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.827768 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:26.867000 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:27.085745 34645 executor.go:111] Tasks: 0 done / 99 total; 51 can run W1208 11:26:27.086159 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:27.086421 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:27.087253 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:27.087535 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:27.088616 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:27.089707 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:27.093941 34645 executor.go:111] Tasks: 51 done / 99 total; 23 can run I1208 11:26:27.100183 34645 executor.go:111] Tasks: 74 done / 99 total; 21 can run I1208 11:26:27.100696 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.100948 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.102107 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.103893 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.104095 34645 executor.go:111] Tasks: 95 done / 99 total; 2 can run I1208 11:26:27.104446 34645 executor.go:111] Tasks: 97 done / 99 total; 2 can run I1208 11:26:27.104639 34645 executor.go:111] Tasks: 99 done / 99 total; 0 can run I1208 11:26:27.166556 34645 target.go:112] Terraform output is in /tmp/test3637098015/out --- PASS: TestDiscoveryFeatureGate (0.37s) === RUN TestVFSServiceAccountIssuerDiscovery I1208 11:26:27.171925 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:27.172191 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:27.172249 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:27.172300 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:27.172355 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:27.172445 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:27.172478 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:27.172517 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:27.172548 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:27.196988 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:27.200112 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.200208 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.239209 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:27.344151 34645 executor.go:111] Tasks: 0 done / 90 total; 45 can run W1208 11:26:27.344450 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:27.344809 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:27.345274 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:27.346524 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:27.345323 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:27.347910 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:27.352319 34645 executor.go:111] Tasks: 45 done / 90 total; 20 can run I1208 11:26:27.357512 34645 executor.go:111] Tasks: 65 done / 90 total; 21 can run I1208 11:26:27.357880 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.358100 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.359257 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.360266 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.360389 34645 executor.go:111] Tasks: 86 done / 90 total; 2 can run I1208 11:26:27.360698 34645 executor.go:111] Tasks: 88 done / 90 total; 2 can run I1208 11:26:27.360877 34645 executor.go:111] Tasks: 90 done / 90 total; 0 can run I1208 11:26:27.416570 34645 target.go:112] Terraform output is in /tmp/test4130905897/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.25s) === RUN TestAWSLBController I1208 11:26:27.420709 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:27.420960 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:27.421015 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:27.421065 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:27.421119 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:27.421213 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:27.421245 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:27.421281 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:27.421311 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:27.446704 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:27.451013 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.451260 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:27.492490 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:28.142654 34645 executor.go:111] Tasks: 0 done / 96 total; 49 can run W1208 11:26:28.143209 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:28.143427 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:28.143572 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:28.143642 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:28.143674 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:28.143774 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:28.146829 34645 executor.go:111] Tasks: 49 done / 96 total; 22 can run I1208 11:26:28.153503 34645 executor.go:111] Tasks: 71 done / 96 total; 21 can run I1208 11:26:28.153676 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.153990 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.154535 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.156871 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.157000 34645 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1208 11:26:28.157401 34645 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1208 11:26:28.157614 34645 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1208 11:26:28.217405 34645 target.go:112] Terraform output is in /tmp/test73405146/out --- PASS: TestAWSLBController (0.81s) === RUN TestManyAddons I1208 11:26:28.229826 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:28.230141 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:28.230209 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:28.230264 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:28.230319 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:28.230412 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:28.230442 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:28.230479 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:28.230510 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:28.256596 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:28.259634 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.259712 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:28.298314 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:29.090523 34645 executor.go:111] Tasks: 0 done / 95 total; 50 can run W1208 11:26:29.090933 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:29.091172 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:29.091225 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:29.091412 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:29.091543 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:29.091629 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:29.100839 34645 executor.go:111] Tasks: 50 done / 95 total; 19 can run I1208 11:26:29.106618 34645 executor.go:111] Tasks: 69 done / 95 total; 22 can run I1208 11:26:29.107034 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.107286 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.108338 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.110889 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.111159 34645 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1208 11:26:29.111854 34645 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1208 11:26:29.112360 34645 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1208 11:26:29.166525 34645 target.go:112] Terraform output is in /tmp/test1720845908/out --- PASS: TestManyAddons (0.95s) === RUN TestManyAddonsCCMIRSA I1208 11:26:29.178342 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:29.178642 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:29.178702 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:29.178821 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:29.178880 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:29.178925 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:29.178954 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:29.178996 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:29.179036 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:29.206757 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:29.210647 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.210742 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:29.250625 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:30.103978 34645 executor.go:111] Tasks: 0 done / 111 total; 59 can run W1208 11:26:30.104332 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:30.105010 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:30.105034 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:30.105064 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:30.105163 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:30.105332 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:30.113247 34645 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1208 11:26:30.119915 34645 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1208 11:26:30.120195 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.120505 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.121478 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.123126 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.123271 34645 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1208 11:26:30.123611 34645 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1208 11:26:30.123882 34645 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1208 11:26:30.197024 34645 target.go:112] Terraform output is in /tmp/test153187285/out --- PASS: TestManyAddonsCCMIRSA (1.03s) === RUN TestManyAddonsCCMIRSA23 I1208 11:26:30.207058 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:30.207322 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:30.207383 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:30.207490 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:30.207546 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:30.207593 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:30.207622 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:30.207662 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:30.207690 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:30.235447 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1208 11:26:30.239043 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.239129 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:30.278298 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:31.154924 34645 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1208 11:26:31.155305 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:31.155362 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:31.155333 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:31.155452 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:31.157229 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:31.157600 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:31.164147 34645 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1208 11:26:31.173619 34645 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1208 11:26:31.173838 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.174129 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.175270 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.177093 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.177241 34645 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1208 11:26:31.177680 34645 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1208 11:26:31.178049 34645 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1208 11:26:31.247691 34645 target.go:112] Terraform output is in /tmp/test1763667067/out --- PASS: TestManyAddonsCCMIRSA23 (1.05s) === RUN TestManyAddonsCCMIRSA24 I1208 11:26:31.257405 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:31.257653 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:31.257713 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:31.257820 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:31.257873 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:31.257917 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:31.257949 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:31.257989 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:31.258019 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:31.285141 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:31.288743 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.288847 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:31.327311 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:32.188483 34645 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1208 11:26:32.189997 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:32.190071 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:32.190236 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:32.188808 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:32.190963 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:32.190921 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:32.197611 34645 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1208 11:26:32.208092 34645 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1208 11:26:32.208371 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.208768 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.210232 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.211283 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.211448 34645 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1208 11:26:32.211780 34645 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1208 11:26:32.212015 34645 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1208 11:26:32.279646 34645 target.go:112] Terraform output is in /tmp/test419214693/out --- PASS: TestManyAddonsCCMIRSA24 (1.03s) === RUN TestManyAddonsCCMIRSA25 I1208 11:26:32.289155 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:32.289402 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:32.289457 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:32.289563 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:32.289621 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:32.289665 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:32.289693 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:32.289731 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:32.289761 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:32.316872 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:32.320348 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.320433 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:32.361728 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:33.249901 34645 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1208 11:26:33.250617 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:33.251807 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:33.251842 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:33.251978 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:33.255972 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:33.256309 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:33.258842 34645 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1208 11:26:33.268531 34645 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1208 11:26:33.268836 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.269151 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.270420 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.271868 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.272002 34645 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1208 11:26:33.272347 34645 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1208 11:26:33.272544 34645 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1208 11:26:33.341817 34645 target.go:112] Terraform output is in /tmp/test1692930315/out --- PASS: TestManyAddonsCCMIRSA25 (1.06s) === RUN TestManyAddonsCCMIRSA26 I1208 11:26:33.352231 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:33.352496 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:33.352561 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:33.352612 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:33.352667 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:33.352762 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:33.352792 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:33.352829 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:33.352859 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:33.385168 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:33.389308 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.389410 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:33.430246 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:34.298506 34645 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1208 11:26:34.299098 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:34.299145 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:34.299274 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:34.299393 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:34.303058 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:34.303178 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:34.308300 34645 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1208 11:26:34.318156 34645 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1208 11:26:34.318510 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.318827 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.320054 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.321713 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.321867 34645 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1208 11:26:34.322243 34645 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1208 11:26:34.322518 34645 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1208 11:26:34.390766 34645 target.go:112] Terraform output is in /tmp/test983382065/out --- PASS: TestManyAddonsCCMIRSA26 (1.05s) === RUN TestCCM I1208 11:26:34.401238 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:34.401492 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:34.401551 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:34.401703 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:34.401778 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:34.401872 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:34.401903 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:34.401941 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:34.401970 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:34.433288 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:34.437592 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.437700 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:34.477441 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:35.311112 34645 executor.go:111] Tasks: 0 done / 96 total; 51 can run I1208 11:26:35.311799 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:35.311894 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.312030 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:35.312078 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.312103 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:35.315186 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:35.319515 34645 executor.go:111] Tasks: 51 done / 96 total; 19 can run I1208 11:26:35.325940 34645 executor.go:111] Tasks: 70 done / 96 total; 22 can run I1208 11:26:35.326163 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.326478 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.327701 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.328928 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.329052 34645 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1208 11:26:35.329359 34645 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1208 11:26:35.329505 34645 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1208 11:26:35.387820 34645 target.go:112] Terraform output is in /tmp/test3975038296/out --- PASS: TestCCM (1.00s) === RUN TestExternalDNS I1208 11:26:35.399451 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:35.399722 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:35.399783 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:35.399837 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:35.399892 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:35.399990 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:35.400021 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:35.400060 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:35.400091 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:35.424236 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:35.427293 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.427393 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.465746 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:35.575264 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:35.575539 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:35.575578 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:35.575679 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:35.575912 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.576022 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:35.576059 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.578548 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:35.584881 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:35.585358 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.585593 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.586655 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.587672 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.587790 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:35.588902 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:35.589256 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:35.643540 34645 target.go:112] Terraform output is in /tmp/test4197448931/out --- PASS: TestExternalDNS (0.25s) === RUN TestExternalDNSIRSA I1208 11:26:35.647535 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:35.647786 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:35.647846 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:35.647902 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:35.647959 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:35.648050 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:35.648083 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:35.648121 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:35.648151 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:35.676462 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:35.679747 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.679850 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.719369 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:35.841560 34645 executor.go:111] Tasks: 0 done / 92 total; 46 can run W1208 11:26:35.841917 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.841993 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:35.842016 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:35.842123 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:35.842476 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:35.842673 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:35.844853 34645 executor.go:111] Tasks: 46 done / 92 total; 21 can run I1208 11:26:35.850317 34645 executor.go:111] Tasks: 67 done / 92 total; 21 can run I1208 11:26:35.850670 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.850850 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.851839 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.853198 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.853320 34645 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1208 11:26:35.853617 34645 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1208 11:26:35.853868 34645 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1208 11:26:35.910509 34645 target.go:112] Terraform output is in /tmp/test231566447/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestKarpenter I1208 11:26:35.914916 34645 featureflag.go:160] FeatureFlag "Karpenter"=true I1208 11:26:35.914962 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:35.915213 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:35.915269 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:35.915319 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:35.915371 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:35.915487 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:35.915519 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:35.915559 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:35.915589 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:35.946566 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:35.953459 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.953564 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.953628 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:35.994968 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:36.189707 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.189877 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.190004 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.309510 34645 executor.go:111] Tasks: 0 done / 108 total; 53 can run I1208 11:26:36.311547 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:36.311580 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:36.311679 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:36.309899 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:36.313763 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:36.315657 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:36.316827 34645 executor.go:111] Tasks: 53 done / 108 total; 26 can run I1208 11:26:36.323179 34645 executor.go:111] Tasks: 79 done / 108 total; 25 can run I1208 11:26:36.323585 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.323739 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.323889 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.325143 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.327384 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.327884 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.329730 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.331001 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.331170 34645 executor.go:111] Tasks: 104 done / 108 total; 2 can run I1208 11:26:36.331515 34645 executor.go:111] Tasks: 106 done / 108 total; 2 can run I1208 11:26:36.331751 34645 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1208 11:26:36.403464 34645 target.go:112] Terraform output is in /tmp/test739573992/out I1208 11:26:36.409921 34645 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.50s) === RUN TestSharedSubnet I1208 11:26:36.410066 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:36.410324 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:36.410387 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:36.410444 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:36.410498 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:36.410599 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:36.410632 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:36.410671 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:36.410701 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:36.435329 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:36.438446 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.438542 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.478501 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:36.584050 34645 executor.go:111] Tasks: 0 done / 80 total; 42 can run W1208 11:26:36.584473 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:36.584478 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:36.584565 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:36.584527 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:36.584658 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:36.584648 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:36.586969 34645 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1208 11:26:36.587431 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:36.592262 34645 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1208 11:26:36.592691 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.592819 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.593792 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.595081 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.595203 34645 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1208 11:26:36.595552 34645 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1208 11:26:36.595774 34645 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1208 11:26:36.643660 34645 target.go:112] Terraform output is in /tmp/test1244810989/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I1208 11:26:36.647515 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:36.647765 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:36.647823 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:36.647874 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:36.647926 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:36.648018 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:36.648048 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:36.648085 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:36.648112 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:36.672164 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:36.675253 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.675347 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.715346 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:36.821020 34645 executor.go:111] Tasks: 0 done / 84 total; 42 can run W1208 11:26:36.821450 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:36.821836 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:36.822054 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:36.822114 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:36.822214 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:36.822199 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:36.823733 34645 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1208 11:26:36.824279 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:36.833838 34645 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1208 11:26:36.834331 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.834561 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.835712 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.836693 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:36.836813 34645 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1208 11:26:36.837161 34645 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1208 11:26:36.837428 34645 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1208 11:26:36.886283 34645 target.go:112] Terraform output is in /tmp/test84388869/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I1208 11:26:36.890586 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:36.891663 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:36.891751 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:36.891818 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:36.891890 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:36.892034 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:36.892094 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:36.892173 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:36.892225 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:36.928060 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:36.931712 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:36.931818 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:36.931884 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:36.931949 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:36.971914 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:37.085112 34645 executor.go:111] Tasks: 0 done / 101 total; 51 can run I1208 11:26:37.085541 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:37.085614 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.085572 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:37.085680 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:37.085724 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.085730 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:37.085905 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:37.086051 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.088837 34645 executor.go:111] Tasks: 51 done / 101 total; 15 can run I1208 11:26:37.098963 34645 executor.go:111] Tasks: 66 done / 101 total; 27 can run I1208 11:26:37.099206 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.099381 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.099721 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.099920 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.100816 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.101451 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.101903 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.102342 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:37.102479 34645 executor.go:111] Tasks: 93 done / 101 total; 4 can run I1208 11:26:37.102908 34645 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1208 11:26:37.103186 34645 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1208 11:26:37.182306 34645 target.go:112] Terraform output is in /tmp/test3163763122/out --- PASS: TestExistingIAM (0.30s) === RUN TestPhaseNetwork I1208 11:26:37.187021 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:37.187306 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:37.187368 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:37.187425 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:37.187483 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:37.187530 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:37.187560 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:37.187602 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:37.187630 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:37.215665 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:37.219038 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.219130 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.219190 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.258693 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:37.368326 34645 executor.go:111] Tasks: 0 done / 119 total; 46 can run I1208 11:26:37.368809 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:37.368915 34645 executor.go:111] Tasks: 46 done / 119 total; 28 can run I1208 11:26:37.369443 34645 executor.go:111] Tasks: 74 done / 119 total; 36 can run I1208 11:26:37.369901 34645 executor.go:111] Tasks: 110 done / 119 total; 5 can run I1208 11:26:37.370100 34645 executor.go:111] Tasks: 115 done / 119 total; 4 can run I1208 11:26:37.370264 34645 executor.go:111] Tasks: 119 done / 119 total; 0 can run I1208 11:26:37.379327 34645 target.go:112] Terraform output is in /tmp/test484532936/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I1208 11:26:37.380119 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:37.380362 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:37.380414 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:37.380462 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:37.380517 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:37.380609 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:37.380640 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:37.380677 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:37.380707 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:37.408984 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:37.412561 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.412742 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.452538 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:37.559481 34645 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1208 11:26:37.559821 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:37.559863 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:37.560011 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:37.559835 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:37.560363 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.560517 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:37.562723 34645 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1208 11:26:37.567998 34645 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1208 11:26:37.568467 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.568688 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.569821 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.570848 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.570972 34645 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1208 11:26:37.571353 34645 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1208 11:26:37.571550 34645 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1208 11:26:37.625710 34645 target.go:112] Terraform output is in /tmp/test3633771151/out --- PASS: TestExternalLoadBalancer (0.25s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I1208 11:26:37.629867 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:37.630116 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:37.630175 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:37.630231 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:37.630286 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:37.630374 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:37.630406 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:37.630445 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:37.630475 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:37.663412 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:37.666762 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.666856 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.666913 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.666970 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.667023 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.667070 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.667123 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.706735 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:37.821872 34645 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1208 11:26:37.822286 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.822689 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:37.822841 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.822733 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:37.822940 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.822942 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1208 11:26:37.823061 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:37.823064 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:37.829398 34645 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1208 11:26:37.834798 34645 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1208 11:26:37.835166 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.835504 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.835801 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.836092 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.836952 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.838630 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.839843 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.841325 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.841889 34645 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1208 11:26:37.843333 34645 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1208 11:26:37.843628 34645 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1208 11:26:37.923508 34645 target.go:112] Terraform output is in /tmp/test123424351/out --- PASS: TestMixedInstancesASG (0.30s) === RUN TestMixedInstancesSpotASG I1208 11:26:37.928617 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:37.928891 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:37.928951 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:37.929007 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:37.929062 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:37.929154 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:37.929183 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:37.929221 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:37.929251 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:37.964812 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:37.968470 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968574 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968637 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968699 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968777 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968832 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:37.968882 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.009544 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:38.125911 34645 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1208 11:26:38.126367 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:38.126719 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:38.126720 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:38.126810 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:38.127199 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:38.127206 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:38.127271 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:38.127413 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:38.129051 34645 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1208 11:26:38.134290 34645 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1208 11:26:38.134756 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.134992 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.135264 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.135559 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.136598 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.140190 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.141551 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.141681 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:38.141856 34645 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1208 11:26:38.143131 34645 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1208 11:26:38.143444 34645 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1208 11:26:38.223099 34645 target.go:112] Terraform output is in /tmp/test3783818214/out --- PASS: TestMixedInstancesSpotASG (0.30s) === RUN TestAdditionalObjects I1208 11:26:38.228301 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:38.228550 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:38.228610 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:38.228721 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:38.228798 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:38.228846 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:38.228876 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:38.228919 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:38.228947 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:38.259036 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1208 11:26:38.262929 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.263041 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.302968 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:38.468920 34645 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1208 11:26:38.469628 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:38.471055 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:38.471672 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:38.472150 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:38.472192 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:38.472353 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:38.477796 34645 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1208 11:26:38.483417 34645 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1208 11:26:38.483825 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.484100 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.485208 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.487246 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1208 11:26:38.487391 34645 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1208 11:26:38.487753 34645 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1208 11:26:38.487954 34645 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1208 11:26:38.544862 34645 target.go:112] Terraform output is in /tmp/test1205422652/out --- PASS: TestAdditionalObjects (0.32s) === RUN TestContainerd I1208 11:26:38.549341 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:38.549601 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:38.549661 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:38.549714 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:38.549770 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:38.549861 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:38.549891 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:38.549928 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:38.549958 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:38.574413 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:38.577443 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.577536 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.615757 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:38.719416 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1208 11:26:38.719914 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:38.719971 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:38.720138 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:38.720173 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:38.720270 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:38.720448 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:38.725823 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:38.732746 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:38.732952 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.733379 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.734065 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.735745 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.735870 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:38.736202 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:38.736394 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:38.791136 34645 target.go:112] Terraform output is in /tmp/test2831691296/out --- PASS: TestContainerd (0.25s) === RUN TestContainerdCustom I1208 11:26:38.795147 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:38.795429 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:38.795489 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:38.795543 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:38.795596 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:38.795687 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:38.795718 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:38.795757 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:38.795790 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:38.821816 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:38.825169 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.825265 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.864358 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:38.973982 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:38.974473 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:38.974652 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:38.974736 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:38.975005 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:38.975038 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:38.975146 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:38.980776 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:38.987141 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:38.987666 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.987893 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.989141 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.990082 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:38.990208 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:38.990509 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:38.990691 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:39.045274 34645 target.go:112] Terraform output is in /tmp/test1849253761/out --- PASS: TestContainerdCustom (0.25s) === RUN TestDockerCustom I1208 11:26:39.049443 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:39.049693 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:39.049752 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:39.049801 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:39.049855 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:39.049947 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:39.049977 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:39.050013 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:39.050043 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:39.075647 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:39.079091 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.079190 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.118237 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:39.221371 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1208 11:26:39.221791 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:39.222274 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:39.222315 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:39.222802 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:39.222773 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:39.222762 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:39.224977 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:39.232293 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:39.232958 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.233229 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.236459 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.239937 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:39.240080 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:39.240417 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:39.240673 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:39.289867 34645 target.go:112] Terraform output is in /tmp/test1208707365/out --- PASS: TestDockerCustom (0.24s) === RUN TestAPIServerNodes I1208 11:26:39.293845 34645 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1208 11:26:39.293876 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:39.294119 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:39.294175 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:39.294226 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:39.294280 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:39.294372 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:39.294403 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:39.294441 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:39.294470 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:39.324547 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1208 11:26:39.328616 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:39.328751 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:39.328824 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.212301 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:40.371580 34645 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1208 11:26:40.373379 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:40.373466 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:40.374355 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:40.372125 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:40.375755 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:40.375892 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:40.375996 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:40.379223 34645 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1208 11:26:40.386190 34645 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1208 11:26:40.386555 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.386746 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.387000 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.387814 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.388248 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.389304 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.389423 34645 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1208 11:26:40.390885 34645 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1208 11:26:40.391349 34645 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1208 11:26:40.457117 34645 target.go:112] Terraform output is in /tmp/test845598572/out I1208 11:26:40.461908 34645 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.17s) === RUN TestNTHQueueProcessor I1208 11:26:40.462048 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:40.462326 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:40.462388 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:40.462440 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:40.462492 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:40.462587 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:40.462618 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:40.462657 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:40.462686 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:40.487611 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1208 11:26:40.490523 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:40.490608 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:41.818764 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:41.949395 34645 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1208 11:26:41.949727 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:41.950371 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1208 11:26:41.950451 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:41.950410 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:41.950538 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:41.950653 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:41.957669 34645 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1208 11:26:41.963293 34645 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1208 11:26:41.963629 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:41.963768 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:41.964698 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:41.966162 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:41.966288 34645 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1208 11:26:41.966661 34645 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1208 11:26:41.966973 34645 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1208 11:26:42.030523 34645 target.go:112] Terraform output is in /tmp/test4100665987/out --- PASS: TestNTHQueueProcessor (1.57s) === RUN TestCustomIRSA I1208 11:26:42.035608 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:42.035854 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:42.035912 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:42.035961 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:42.036014 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:42.036120 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:42.036155 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:42.036192 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:42.036223 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:42.069352 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1208 11:26:42.072859 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.072962 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.116432 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:42.660172 34645 executor.go:111] Tasks: 0 done / 100 total; 50 can run I1208 11:26:42.660541 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:42.660902 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:42.660954 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:42.661095 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1208 11:26:42.661116 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:42.661139 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:42.668981 34645 executor.go:111] Tasks: 50 done / 100 total; 25 can run I1208 11:26:42.674551 34645 executor.go:111] Tasks: 75 done / 100 total; 21 can run I1208 11:26:42.675006 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.675211 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.676153 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.677449 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.677569 34645 executor.go:111] Tasks: 96 done / 100 total; 2 can run I1208 11:26:42.678050 34645 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1208 11:26:42.678356 34645 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1208 11:26:42.739916 34645 target.go:112] Terraform output is in /tmp/test1961986290/out --- PASS: TestCustomIRSA (0.71s) === RUN TestClusterNameDigit I1208 11:26:42.748078 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:42.748463 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:42.748573 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:42.748672 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:42.748779 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:42.748939 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:42.748999 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:42.749073 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:42.749133 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:42.776579 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1208 11:26:42.779882 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.779974 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.818715 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:42.927754 34645 executor.go:111] Tasks: 0 done / 95 total; 47 can run I1208 11:26:42.929704 34645 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1208 11:26:42.929762 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:42.929942 34645 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1208 11:26:42.929924 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1208 11:26:42.930298 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target W1208 11:26:42.930328 34645 warmpool.go:124] ASG warm pool is not supported by the terraform target I1208 11:26:42.935877 34645 executor.go:111] Tasks: 47 done / 95 total; 23 can run I1208 11:26:42.941279 34645 executor.go:111] Tasks: 70 done / 95 total; 21 can run I1208 11:26:42.941661 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.941960 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.943390 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.944852 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:42.944992 34645 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1208 11:26:42.945290 34645 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1208 11:26:42.945552 34645 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1208 11:26:43.004273 34645 target.go:112] Terraform output is in /tmp/test4211271366/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I1208 11:26:43.008802 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:43.009055 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:43.009115 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:43.009169 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:43.009225 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:43.009268 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:43.009297 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:43.009335 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:43.009366 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:43.009439 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I1208 11:26:43.037271 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:43.040358 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.040464 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1208 11:26:43.080437 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:43.188558 34645 iamrole.go:89] ListRoles: { } I1208 11:26:43.190341 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:43.190694 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.190876 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.191141 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I1208 11:26:43.191149 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.191257 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I1208 11:26:43.191282 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1208 11:26:43.191443 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I1208 11:26:43.191427 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:43.191441 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.192461 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1208 11:26:43.192599 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:43.192717 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.192999 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:43.193258 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:43.193358 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:43.193593 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:43.193744 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:43.193839 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:43.193909 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:43.193988 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.194079 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:43.194394 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:43.194553 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.196763 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:43.197715 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1208 11:26:43.197773 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.197887 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1208 11:26:43.198056 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1208 11:26:43.197963 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1208 11:26:43.198188 34645 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1208 11:26:43.198289 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:43.198420 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.198415 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1208 11:26:43.198613 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.198944 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.199103 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:43.198950 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1208 11:26:43.199216 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:43.199297 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:43.199268 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1208 11:26:43.199365 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1208 11:26:43.199564 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.199765 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:43.199958 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:43.200058 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:43.200138 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:43.200314 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:43.200397 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:43.200455 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1208 11:26:43.200546 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:43.200596 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:43.200656 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:43.200695 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I1208 11:26:43.200924 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1208 11:26:43.203656 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:43.203951 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.204143 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1208 11:26:43.204449 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I1208 11:26:43.204636 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.204844 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:43.204926 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1208 11:26:43.205144 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.205314 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.205472 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.205698 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.205883 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.206106 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.206255 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.206342 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.206411 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1208 11:26:43.206564 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I1208 11:26:43.206714 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.206845 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.206951 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.207048 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:43.207140 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1208 11:26:43.207311 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:43.207355 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1208 11:26:43.207396 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.207532 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1208 11:26:43.208012 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:43.208274 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:43.208390 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:43.208439 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:43.208496 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:43.208559 34645 dns.go:238] Pre-creating DNS records I1208 11:26:43.208592 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:43.214293 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:43.217233 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.217333 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1208 11:26:43.256677 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:43.365452 34645 iamrole.go:89] ListRoles: { } I1208 11:26:43.367212 34645 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1208 11:26:43.367414 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:43.367694 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.367876 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.368137 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.369391 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.369578 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:43.369620 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:43.369697 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.369742 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:43.369834 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:43.373830 34645 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1208 11:26:43.374633 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.374824 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I1208 11:26:43.375010 34645 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1208 11:26:43.375038 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:43.375068 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1208 11:26:43.375124 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1208 11:26:43.375304 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:43.375364 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.375475 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:43.375569 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1208 11:26:43.375733 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:43.375805 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.380350 34645 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1208 11:26:43.380557 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.380862 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.381072 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.381257 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.381535 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.381769 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.382035 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:43.384272 34645 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1208 11:26:43.384580 34645 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1208 11:26:43.384730 34645 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1208 11:26:43.387524 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:43.387588 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:43.387624 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:43.387656 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:43.387690 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:43.387728 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:43.387761 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:43.387790 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:43.387823 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:43.387857 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:43.387891 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:43.387932 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:43.387969 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:43.388002 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:43.388034 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:43.388071 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:43.388103 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:43.388137 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:43.388172 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:43.388212 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:43.388244 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:43.388276 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:43.388307 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:43.388344 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:43.388382 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:43.388417 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:43.389790 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:43.389821 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:43.389864 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.389914 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1208 11:26:43.389970 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390030 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390080 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390136 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390184 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390224 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1208 11:26:43.390269 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390320 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1208 11:26:43.390360 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W1208 11:26:43.390406 34645 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W1208 11:26:43.390440 34645 api.go:53] PageSize not implemented I1208 11:26:43.390465 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:43.390499 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:43.390518 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:43.390550 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:43.390570 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:43.390624 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:43.390652 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:43.390699 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:43.390742 34645 iamrole.go:89] ListRoles: { } I1208 11:26:43.390802 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:43.390850 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:43.390885 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:43.391412 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I1208 11:26:43.391535 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1208 11:26:43.391521 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1208 11:26:43.391666 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } I1208 11:26:43.391497 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } internet-gateway:igw-2 ok route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok I1208 11:26:43.391660 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I1208 11:26:43.391766 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1208 11:26:43.391839 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I1208 11:26:43.391748 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1208 11:26:43.391920 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1208 11:26:43.391986 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:43.392052 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1208 11:26:43.392233 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:43.392257 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1208 11:26:43.392315 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:43.392363 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1208 11:26:43.392548 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:43.392561 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:43.392639 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:43.392662 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I1208 11:26:43.392739 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I1208 11:26:43.392758 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1208 11:26:43.392803 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I1208 11:26:43.392841 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I1208 11:26:43.392885 34645 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I1208 11:26:43.392897 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1208 11:26:43.392926 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I1208 11:26:43.392992 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:43.393070 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:43.393142 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:43.393413 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.38s) === RUN TestLifecycleMinimalOpenstack I1208 11:26:43.393691 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I1208 11:26:43.403449 34645 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:43.625587 34645 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1208 11:26:43.626764 34645 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1208 11:26:43.642968 34645 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1208 11:26:43.652767 34645 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1208 11:26:43.679942 34645 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1208 11:26:43.683308 34645 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1208 11:26:43.686431 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:43.686435 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:43.686632 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1208 11:26:43.692162 34645 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:43.888364 34645 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1208 11:26:43.888414 34645 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1208 11:26:43.908168 34645 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1208 11:26:43.916211 34645 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1208 11:26:43.926914 34645 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1208 11:26:43.929627 34645 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1208 11:26:43.931884 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:43.932677 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:43.933557 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1208 11:26:43.934411 34645 executor.go:111] Tasks: 96 done / 96 total; 0 can run SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:a1103b91-bf9e-4cb2-b528-f907ef847c2c ok ServerGroup:af509395-3499-4536-93df-aa461dcf5974 ok Instance:18c87a93-3dde-4be2-9195-5b408f52ebc2 ok SecurityGroup:e65d6a93-dc6c-4125-909a-98afefc9f602 ok Port:e06990b9-0b9e-41ab-8aac-eaffa62f7c88 ok Subnet:ac95fc15-3991-46aa-9c91-0c93364c0eb2 ok Network:d0a1913a-86f5-4f6b-b18d-19d0cd8c6d84 ok SecurityGroup:75514434-3a36-40dc-a9c5-e44b193dd5df ok SecurityGroup:17a39f5d-503f-4ced-b621-ba7627c37e54 ok Port:dc348514-99e1-4fd6-8c65-04e8193bae34 ok Instance:e3abdc22-c47e-4bb1-9995-a86a723714aa ok Port:f9f7f800-5e01-4a01-be6d-c54db07347ea ok ServerGroup:74949f7b-9d41-4422-9c49-330ec2460575 ok --- PASS: TestLifecycleMinimalOpenstack (1.64s) === RUN TestLifecycleMinimalGCE I1208 11:26:45.035770 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I1208 11:26:45.037638 34645 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1208 11:26:45.275796 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:45.275889 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:45.275996 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1208 11:26:45.276032 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1208 11:26:45.276055 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:45.276086 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:45.276157 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:45.277982 34645 executor.go:111] Tasks: 0 done / 64 total; 41 can run W1208 11:26:45.281472 34645 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I1208 11:26:45.281547 34645 executor.go:111] Tasks: 40 done / 64 total; 18 can run I1208 11:26:45.286886 34645 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1208 11:26:45.289735 34645 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1208 11:26:45.289974 34645 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1208 11:26:45.290083 34645 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1208 11:26:45.514644 34645 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1208 11:26:45.514733 34645 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1208 11:26:45.514838 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1208 11:26:45.514874 34645 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1208 11:26:45.514901 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:45.514938 34645 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1208 11:26:45.515016 34645 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1208 11:26:45.516860 34645 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1208 11:26:45.520101 34645 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1208 11:26:45.526831 34645 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1208 11:26:45.529745 34645 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1208 11:26:45.529928 34645 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1208 11:26:45.531546 34645 gce.go:98] Scanning zones: [us-test1-a] I1208 11:26:45.531938 34645 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I1208 11:26:45.531963 34645 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:node-to-node-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1670498805 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1670498805 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.50s) === RUN TestLifecycleFloatingIPOpenstack I1208 11:26:45.533596 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I1208 11:26:45.545313 34645 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:45.747852 34645 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1208 11:26:45.747898 34645 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1208 11:26:45.766102 34645 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1208 11:26:45.770585 34645 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1208 11:26:45.807254 34645 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1208 11:26:45.810473 34645 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1208 11:26:45.812981 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:45.813148 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:45.813338 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1208 11:26:45.818055 34645 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1208 11:26:46.013475 34645 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1208 11:26:46.013528 34645 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1208 11:26:46.037655 34645 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1208 11:26:46.041483 34645 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1208 11:26:46.050854 34645 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1208 11:26:46.054864 34645 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1208 11:26:46.057859 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:46.058640 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1208 11:26:46.058824 34645 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1208 11:26:46.060013 34645 executor.go:111] Tasks: 98 done / 98 total; 0 can run Subnet:55ff79ca-0a9a-4cac-a477-c1370001829e ok SecurityGroup:3bd91367-30e3-4ed6-a0c8-9f760beaf021 ok Port:2413df79-9891-413d-8a91-815773488e22 ok Port:6979e749-8270-42d9-b42e-2d3acfaf45fd ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok ServerGroup:73a5a3c9-77e8-45c6-8e60-8a970e63a0ac ok Router-IF:55ff79ca-0a9a-4cac-a477-c1370001829e ok ServerGroup:359dc238-9626-450a-a31b-dca47905a290 ok Port:2b8641f3-7afa-4221-8085-8094d13b67e6 ok Router:0f84007e-c684-46b4-b87c-1ecf254512a4 ok SecurityGroup:ee6b7cb3-7575-43c9-8027-c66cc7c97e90 ok Instance:046322ea-7e07-45c1-a04a-dc6f6d13e52b ok Instance:0d0c6e39-d341-4da3-bed5-0f2dfa8a8508 ok Port:98f9962a-b815-4867-8897-502e88f6b15b ok Network:eb1091eb-9a24-43f4-bd99-f9ca0886a653 ok Instance:0d68c21a-3fe8-4c28-be10-a70e0e0deda8 ok SecurityGroup:2b079c83-aeaf-4a4c-a547-24a94c428020 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.63s) === RUN TestLifecyclePrivateCalico I1208 11:26:47.164631 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:47.164918 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:47.164975 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:47.165024 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:47.165078 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:47.165120 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:47.165149 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:47.165184 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:47.165211 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:47.165284 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I1208 11:26:47.194221 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:47.198017 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.198117 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.198180 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.237921 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:47.704323 34645 iamrole.go:89] ListRoles: { } I1208 11:26:47.706952 34645 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1208 11:26:47.707440 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I1208 11:26:47.707620 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1208 11:26:47.707668 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.707819 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I1208 11:26:47.707901 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:47.707951 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:47.707982 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:47.707966 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1208 11:26:47.708206 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:47.708323 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:47.708361 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.708364 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I1208 11:26:47.708456 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1208 11:26:47.708492 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:47.708575 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:47.708610 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.708741 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.708849 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:47.708899 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.709073 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.709094 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.709462 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:47.709527 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:47.709578 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:47.709660 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:47.710112 34645 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1208 11:26:47.710250 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1208 11:26:47.710375 34645 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1208 11:26:47.710306 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1208 11:26:47.710446 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:47.710554 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1208 11:26:47.710560 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:47.710588 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1208 11:26:47.710676 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:47.710748 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1208 11:26:47.710734 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1208 11:26:47.710894 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1208 11:26:47.710936 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:47.711035 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:47.711008 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1208 11:26:47.711085 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:47.711180 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.711312 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1208 11:26:47.711461 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:47.711459 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1208 11:26:47.711709 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.711750 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1208 11:26:47.711908 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1208 11:26:47.711917 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I1208 11:26:47.712105 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:47.712182 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:47.712222 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:47.712312 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:47.712358 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:47.712396 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:47.712511 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.712610 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.712695 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:47.712742 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:47.712829 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:47.712779 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1208 11:26:47.712931 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1208 11:26:47.712659 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1208 11:26:47.713066 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.713195 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:47.713255 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:47.713306 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:47.713347 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:47.713496 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:47.713499 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1208 11:26:47.713570 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1208 11:26:47.713663 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:47.713706 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I1208 11:26:47.713802 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:47.713853 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:47.713894 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:47.713987 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:47.714042 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:47.714085 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:47.714135 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:47.715638 34645 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1208 11:26:47.715789 34645 api.go:53] PageSize not implemented I1208 11:26:47.715856 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.715959 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1208 11:26:47.716031 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:47.716120 34645 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1208 11:26:47.716188 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1208 11:26:47.716176 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:47.716221 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1208 11:26:47.716298 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } W1208 11:26:47.716395 34645 api.go:53] PageSize not implemented I1208 11:26:47.716365 34645 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.716266 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" }] }] } I1208 11:26:47.716429 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1208 11:26:47.716608 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1208 11:26:47.716650 34645 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1208 11:26:47.716713 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1208 11:26:47.716745 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1208 11:26:47.716793 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1208 11:26:47.716589 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1208 11:26:47.716890 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.717039 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.717169 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.717400 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.717515 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.717622 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:47.717762 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:47.717902 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:47.717962 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:47.718040 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:47.718140 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.718277 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.718418 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.718552 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:47.718594 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.718753 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1208 11:26:47.718875 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.719023 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.719160 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.719294 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:47.719424 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1208 11:26:47.719463 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.719596 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1208 11:26:47.719692 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:47.719819 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.719960 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:47.719999 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" }] }] } I1208 11:26:47.720137 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.720217 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:47.720334 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1208 11:26:47.720425 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.720556 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.720683 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.720736 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1208 11:26:47.720826 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" }] }] } I1208 11:26:47.720959 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1208 11:26:47.721091 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:47.721195 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1208 11:26:47.721230 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1208 11:26:47.721258 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.724135 34645 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1208 11:26:47.724237 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:47.724281 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:47.724324 34645 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:47.724458 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:47.724619 34645 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1208 11:26:47.724742 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:47.724792 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:47.724840 34645 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1208 11:26:47.724909 34645 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1208 11:26:47.725022 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:47.725164 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1208 11:26:47.725235 34645 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1208 11:26:47.725280 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:47.725312 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:47.725360 34645 dns.go:238] Pre-creating DNS records I1208 11:26:47.725385 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:47.731820 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1208 11:26:47.735470 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.735565 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.735630 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:47.774375 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.241304 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.243171 34645 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1208 11:26:48.243537 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I1208 11:26:48.243863 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.243929 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:48.244093 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.244335 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.244516 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:48.244570 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:48.244665 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.244732 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.244839 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.245986 34645 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1208 11:26:48.246173 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1208 11:26:48.246128 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.246322 34645 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1208 11:26:48.246389 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1208 11:26:48.246299 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1208 11:26:48.246463 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1208 11:26:48.246588 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.246631 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:48.246667 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.246759 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.246836 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1208 11:26:48.247813 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.248178 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1208 11:26:48.248415 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.248500 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.248650 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:48.248708 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1208 11:26:48.249049 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.249136 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.249246 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:48.249308 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:48.249622 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.256376 34645 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1208 11:26:48.256531 34645 api.go:53] PageSize not implemented I1208 11:26:48.256583 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1208 11:26:48.256650 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.256744 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1208 11:26:48.256819 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1208 11:26:48.256794 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W1208 11:26:48.256970 34645 api.go:53] PageSize not implemented I1208 11:26:48.257039 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.257004 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1208 11:26:48.257172 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1208 11:26:48.257240 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I1208 11:26:48.257281 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.257413 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.257607 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.257918 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.258086 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.258252 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.258332 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.258511 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.258705 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:48.258808 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1208 11:26:48.258851 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1208 11:26:48.260022 34645 executor.go:111] Tasks: 118 done / 127 total; 5 can run W1208 11:26:48.260161 34645 api.go:53] PageSize not implemented I1208 11:26:48.260146 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.260112 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.260197 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1208 11:26:48.260284 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:48.260233 34645 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W1208 11:26:48.260324 34645 api.go:53] PageSize not implemented I1208 11:26:48.260417 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1208 11:26:48.260616 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1208 11:26:48.260733 34645 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1208 11:26:48.260813 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.260936 34645 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1208 11:26:48.265165 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:48.265232 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:48.265271 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:48.265310 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:48.265344 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:48.265383 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1208 11:26:48.265439 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:48.265496 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:48.265557 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:48.265616 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:48.265692 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:48.265757 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:48.265834 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:48.265906 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1208 11:26:48.265980 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:48.266049 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:48.266118 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:48.266180 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:48.266226 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:48.266260 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:48.266297 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:48.266333 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:48.266369 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:48.266404 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:48.266439 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:48.266475 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.266513 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:48.266546 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:48.266584 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:48.266617 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:48.266652 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:48.266689 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:48.266728 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:48.266764 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:48.268432 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:48.268466 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:48.268509 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268576 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1208 11:26:48.268656 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268715 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268785 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268840 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268881 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.268926 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1208 11:26:48.268985 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.269041 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:48.269069 34645 natgateway.go:117] DescribeNatGateways: { } I1208 11:26:48.269107 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1208 11:26:48.269148 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W1208 11:26:48.269202 34645 api.go:53] PageSize not implemented I1208 11:26:48.269221 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1208 11:26:48.269276 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:48.269304 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:48.269325 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:48.269350 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:48.269368 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:48.269423 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.269454 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.269497 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:48.269543 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.269591 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:48.269639 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:48.269680 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1208 11:26:48.269718 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:48.269754 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1208 11:26:48.270418 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I1208 11:26:48.270463 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1208 11:26:48.270516 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } load-balancer:bastion-privatecalico-exa-hocohm ok I1208 11:26:48.270534 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I1208 11:26:48.270605 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:48.270625 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok autoscaling-group:bastion.privatecalico.example.com ok I1208 11:26:48.270629 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I1208 11:26:48.270652 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1208 11:26:48.270661 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1208 11:26:48.270712 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:nodes.privatecalico.example.com ok I1208 11:26:48.270582 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I1208 11:26:48.270748 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I1208 11:26:48.270864 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1208 11:26:48.270924 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I1208 11:26:48.270803 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1208 11:26:48.271078 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.271102 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1208 11:26:48.271138 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.271197 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W1208 11:26:48.271311 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.271340 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:48.271382 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:48.271457 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1208 11:26:48.271564 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.271586 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I1208 11:26:48.271645 34645 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1208 11:26:48.271716 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:48.271822 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I1208 11:26:48.271869 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1208 11:26:48.271894 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I1208 11:26:48.271974 34645 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I1208 11:26:48.271995 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1208 11:26:48.271979 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:48.272103 34645 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1208 11:26:48.272031 34645 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I1208 11:26:48.272191 34645 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I1208 11:26:48.272177 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.272209 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1208 11:26:48.272240 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I1208 11:26:48.272279 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I1208 11:26:48.272323 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I1208 11:26:48.272336 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1208 11:26:48.272365 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I1208 11:26:48.272238 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } W1208 11:26:48.272401 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.272427 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:48.272466 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.272516 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1208 11:26:48.272616 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.272631 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:48.272711 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:48.272772 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1208 11:26:48.272847 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:48.272923 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:48.273228 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.11s) === RUN TestLifecyclePrivateKopeio I1208 11:26:48.273556 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:48.273871 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:48.273919 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:48.273961 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:48.274010 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:48.274053 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:48.274085 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:48.274123 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:48.274151 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:48.274218 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I1208 11:26:48.303600 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:48.307012 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.307101 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.307163 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.347237 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.465236 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.467833 34645 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1208 11:26:48.468425 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:48.468552 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.468665 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I1208 11:26:48.468768 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.468820 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I1208 11:26:48.468854 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1208 11:26:48.469027 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.469106 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.469267 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:48.469229 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I1208 11:26:48.469343 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:48.469409 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:48.469366 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1208 11:26:48.469490 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.469505 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1208 11:26:48.469556 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.469617 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1208 11:26:48.469815 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.470024 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.470186 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:48.470371 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.470513 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:48.470602 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:48.470679 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:48.470818 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:48.470944 34645 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1208 11:26:48.471051 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.471134 34645 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1208 11:26:48.471180 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1208 11:26:48.471342 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1208 11:26:48.471352 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.471324 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I1208 11:26:48.471459 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1208 11:26:48.471481 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.471643 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:48.471621 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1208 11:26:48.471731 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1208 11:26:48.471710 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1208 11:26:48.471863 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1208 11:26:48.471986 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1208 11:26:48.471962 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.473165 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1208 11:26:48.472130 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1208 11:26:48.473450 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.473672 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.473686 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1208 11:26:48.473843 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1208 11:26:48.473915 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I1208 11:26:48.474024 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I1208 11:26:48.474084 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.474250 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.474401 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1208 11:26:48.474622 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.474822 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.474983 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1208 11:26:48.475208 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.475343 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:48.475422 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.475513 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:48.475715 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.475592 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:48.476029 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.477043 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:48.477106 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.477187 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.477270 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:48.477329 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I1208 11:26:48.478076 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.478248 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:48.478261 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I1208 11:26:48.478325 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:48.478392 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:48.478253 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1208 11:26:48.478437 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:48.478477 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1208 11:26:48.478519 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:48.478567 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:48.478605 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:48.478676 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:48.478721 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1208 11:26:48.478796 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1208 11:26:48.478836 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.478419 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1208 11:26:48.478890 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:48.478936 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I1208 11:26:48.479037 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1208 11:26:48.479084 34645 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I1208 11:26:48.479174 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I1208 11:26:48.479306 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:48.479353 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:48.479395 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.479445 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:48.479502 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:48.480584 34645 executor.go:111] Tasks: 78 done / 126 total; 39 can run W1208 11:26:48.480893 34645 api.go:53] PageSize not implemented W1208 11:26:48.480945 34645 api.go:53] PageSize not implemented I1208 11:26:48.480841 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.481046 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1208 11:26:48.481026 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I1208 11:26:48.481170 34645 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I1208 11:26:48.481287 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1208 11:26:48.481342 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1208 11:26:48.481445 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1208 11:26:48.481585 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.481330 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.481694 34645 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1208 11:26:48.481786 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.481840 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1208 11:26:48.481745 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I1208 11:26:48.481919 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1208 11:26:48.482164 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:48.482319 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.482418 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1208 11:26:48.482560 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.482675 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1208 11:26:48.482786 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.482878 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.483011 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.483141 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.483254 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.483380 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.483530 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1208 11:26:48.483632 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.483774 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.483910 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.484042 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:48.484083 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:48.484140 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.484386 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I1208 11:26:48.484494 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.484750 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:48.484979 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:48.485048 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.485193 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.485376 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.485550 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I1208 11:26:48.485604 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1208 11:26:48.485734 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.485910 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I1208 11:26:48.485970 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.486128 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.486241 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.486417 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I1208 11:26:48.486540 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1208 11:26:48.486578 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1208 11:26:48.486710 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1208 11:26:48.486745 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:48.486851 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" }] }] } I1208 11:26:48.486987 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.487036 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1208 11:26:48.487161 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1208 11:26:48.487198 34645 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1208 11:26:48.487266 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I1208 11:26:48.487312 34645 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1208 11:26:48.487405 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.487491 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1208 11:26:48.487623 34645 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1208 11:26:48.487691 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:48.487754 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I1208 11:26:48.487815 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1208 11:26:48.487874 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:48.487946 34645 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1208 11:26:48.487997 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:48.488039 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1208 11:26:48.488126 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.488280 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1208 11:26:48.489661 34645 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1208 11:26:48.489752 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.489931 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-1" } I1208 11:26:48.489898 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.490011 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.490124 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.490213 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I1208 11:26:48.490312 34645 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1208 11:26:48.490543 34645 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1208 11:26:48.490599 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.490637 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.490690 34645 dns.go:238] Pre-creating DNS records I1208 11:26:48.490720 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.497335 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:48.500769 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.500868 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.500939 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.540733 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.662019 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.664417 34645 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1208 11:26:48.664776 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:48.664848 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.664972 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.665177 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.665457 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.665631 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:48.665676 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:48.665724 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.666002 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.666233 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.667705 34645 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1208 11:26:48.667846 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1208 11:26:48.667966 34645 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1208 11:26:48.668016 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1208 11:26:48.668124 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.668282 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.668476 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:48.668523 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1208 11:26:48.668685 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:48.668765 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.668949 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1208 11:26:48.669129 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.669194 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.669457 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1208 11:26:48.669531 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1208 11:26:48.669744 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1208 11:26:48.669947 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1208 11:26:48.670145 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.670322 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.670459 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.670616 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.670763 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.670912 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.670995 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.671040 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.671133 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:48.671248 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:48.671317 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:48.674289 34645 executor.go:111] Tasks: 78 done / 126 total; 39 can run W1208 11:26:48.674466 34645 api.go:53] PageSize not implemented I1208 11:26:48.674523 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.674505 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.674645 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.674624 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.674719 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } W1208 11:26:48.674865 34645 api.go:53] PageSize not implemented I1208 11:26:48.674769 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.674899 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.674989 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.675022 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1208 11:26:48.675188 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1208 11:26:48.675475 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1208 11:26:48.675674 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.675857 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1208 11:26:48.675957 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.676068 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.676197 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.676487 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1208 11:26:48.676685 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.676810 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.676971 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:48.679051 34645 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1208 11:26:48.679151 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.679155 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.679236 34645 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" W1208 11:26:48.679313 34645 api.go:53] PageSize not implemented I1208 11:26:48.679307 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.679362 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.679586 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W1208 11:26:48.679657 34645 api.go:53] PageSize not implemented I1208 11:26:48.679680 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.679866 34645 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1208 11:26:48.680067 34645 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1208 11:26:48.683804 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:48.683875 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:48.683914 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:48.683952 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:48.683992 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:48.684035 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:48.684071 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:48.684105 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:48.684141 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:48.684179 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:48.684217 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:48.684258 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:48.684301 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1208 11:26:48.684348 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:48.684385 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:48.684421 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:48.684462 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:48.684497 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:48.684535 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1208 11:26:48.684573 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:48.684611 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:48.684648 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:48.684686 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:48.684724 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:48.684763 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.684802 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:48.684838 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:48.684877 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1208 11:26:48.684917 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1208 11:26:48.684957 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:48.684994 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:48.685028 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:48.685066 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:48.685108 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:48.685145 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:48.686907 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:48.686944 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:48.686991 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687061 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1208 11:26:48.687141 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687207 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687256 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687312 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687354 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687409 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1208 11:26:48.687478 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687551 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:48.687584 34645 natgateway.go:117] DescribeNatGateways: { } I1208 11:26:48.687622 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1208 11:26:48.687663 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W1208 11:26:48.687718 34645 api.go:53] PageSize not implemented I1208 11:26:48.687739 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1208 11:26:48.687799 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:48.687829 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:48.687847 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:48.687874 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:48.687893 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:48.687946 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.687973 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.688022 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:48.688064 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.688107 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:48.688158 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:48.688210 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-1","rtb-2"] } I1208 11:26:48.688256 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1208 11:26:48.688292 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1208 11:26:48.688942 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1208 11:26:48.689013 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I1208 11:26:48.688987 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-config:lt-2 ok I1208 11:26:48.689069 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } autoscaling-group:bastion.privatekopeio.example.com ok I1208 11:26:48.689107 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:48.689187 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I1208 11:26:48.689177 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } load-balancer:api-privatekopeio-example-tl2bv8 ok I1208 11:26:48.689221 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I1208 11:26:48.689313 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok W1208 11:26:48.689286 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.689083 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.689378 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I1208 11:26:48.689429 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I1208 11:26:48.689399 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1208 11:26:48.689487 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1208 11:26:48.689557 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:48.689590 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:48.689649 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W1208 11:26:48.689861 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.689887 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-1 ok I1208 11:26:48.689937 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.689992 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W1208 11:26:48.690104 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.690119 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:48.690145 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-3 ok I1208 11:26:48.690201 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:48.690331 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.690341 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I1208 11:26:48.690423 34645 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1208 11:26:48.690440 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1208 11:26:48.690476 34645 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I1208 11:26:48.690381 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W1208 11:26:48.690528 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.690563 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:48.690526 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I1208 11:26:48.690622 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1208 11:26:48.690603 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1208 11:26:48.690647 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1208 11:26:48.690682 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } I1208 11:26:48.690672 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } iam-role:nodes.privatekopeio.example.com ok subnet:subnet-4 ok I1208 11:26:48.690719 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:48.690727 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I1208 11:26:48.690805 34645 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I1208 11:26:48.690821 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1208 11:26:48.690852 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I1208 11:26:48.690784 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1208 11:26:48.690912 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.690968 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1208 11:26:48.691056 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:48.691069 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:48.691144 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I1208 11:26:48.691234 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1208 11:26:48.691295 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:48.691407 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:48.691510 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:48.691811 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.42s) === RUN TestLifecycleIPv6 I1208 11:26:48.692181 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:48.692466 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:48.692511 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:48.692556 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:48.692603 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:48.692643 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:48.692674 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:48.692710 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:48.692735 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:48.692800 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I1208 11:26:48.723690 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:48.727610 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.727705 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.767352 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.946719 34645 iamrole.go:89] ListRoles: { } I1208 11:26:48.948546 34645 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1208 11:26:48.949037 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I1208 11:26:48.949200 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.949397 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I1208 11:26:48.949463 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:48.949480 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:48.950520 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:48.950811 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:48.951368 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:48.949597 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1208 11:26:48.951866 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.952049 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1208 11:26:48.952111 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.952211 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1208 11:26:48.952312 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.952402 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:48.952410 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.952647 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:48.952742 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:48.952826 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.953110 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:48.953260 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.953551 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:48.953898 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.956295 34645 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1208 11:26:48.956516 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1208 11:26:48.956517 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1208 11:26:48.956648 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1208 11:26:48.956727 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.956797 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1208 11:26:48.956863 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.957028 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:48.956899 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1208 11:26:48.957106 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:48.957096 34645 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1208 11:26:48.957173 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.957287 34645 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1208 11:26:48.957330 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:48.957426 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:48.957380 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1208 11:26:48.957519 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1208 11:26:48.957521 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1208 11:26:48.957815 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:48.958050 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.958200 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I1208 11:26:48.958293 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.958373 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I1208 11:26:48.958526 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:48.958592 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1208 11:26:48.958709 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:48.958835 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:48.958911 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:48.958956 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:48.958997 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.959070 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:48.959113 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] } I1208 11:26:48.959198 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:48.959242 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1208 11:26:48.959314 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:48.959394 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:48.959436 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:48.959436 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1208 11:26:48.962403 34645 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1208 11:26:48.962668 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.962815 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.962986 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:48.963131 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I1208 11:26:48.963462 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1208 11:26:48.963622 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.963839 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I1208 11:26:48.964019 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:48.964159 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.964266 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1208 11:26:48.964302 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.964377 34645 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I1208 11:26:48.964512 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:48.964660 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1208 11:26:48.964865 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:48.964917 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.965038 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1208 11:26:48.965278 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.965533 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1208 11:26:48.965802 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.966051 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.966234 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1208 11:26:48.966341 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.966505 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.966559 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:48.966611 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.966804 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1208 11:26:48.966944 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1208 11:26:48.967048 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:48.967098 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1208 11:26:48.967243 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.967382 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.967513 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.967622 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.967699 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.967923 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:48.967982 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.968165 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.971133 34645 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1208 11:26:48.971274 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.971407 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.971520 34645 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.971411 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:48.971603 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:48.971657 34645 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1208 11:26:48.971726 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:48.971807 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1208 11:26:48.971745 34645 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I1208 11:26:48.971869 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:48.971882 34645 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I1208 11:26:48.971934 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:48.971981 34645 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1208 11:26:48.971975 34645 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1208 11:26:48.972106 34645 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1208 11:26:48.972183 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:48.972321 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1208 11:26:48.972337 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.972536 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.973039 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.973166 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:48.973284 34645 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1208 11:26:48.973340 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:48.973372 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.973422 34645 dns.go:238] Pre-creating DNS records I1208 11:26:48.973451 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:48.979470 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:48.982985 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:48.983079 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.025510 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.205456 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.207545 34645 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1208 11:26:49.207736 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.207997 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:49.207986 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.208187 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.208262 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.208345 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.208476 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.208546 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:49.208568 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:49.208597 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.209904 34645 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1208 11:26:49.210029 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.210139 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1208 11:26:49.210147 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.210268 34645 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1208 11:26:49.210263 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.210290 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1208 11:26:49.210370 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1208 11:26:49.210337 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1208 11:26:49.210454 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.210502 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1208 11:26:49.210722 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:49.210828 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1208 11:26:49.211034 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.211069 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.211226 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.215175 34645 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1208 11:26:49.215352 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.215426 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:49.215489 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1208 11:26:49.215536 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1208 11:26:49.215584 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I1208 11:26:49.215757 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:49.215850 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.215998 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.216198 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.216279 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.216364 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.216406 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.216536 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.216777 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.220654 34645 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1208 11:26:49.220768 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.220860 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:49.220834 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:49.220942 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:49.220967 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1208 11:26:49.220974 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.221045 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1208 11:26:49.221117 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1208 11:26:49.221177 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1208 11:26:49.221215 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1208 11:26:49.221388 34645 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1208 11:26:49.221580 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:49.221608 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.221697 34645 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1208 11:26:49.221725 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:49.221765 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1208 11:26:49.221778 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:49.221972 34645 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1208 11:26:49.222006 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:49.222035 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1208 11:26:49.222411 34645 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1208 11:26:49.227380 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:49.227468 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:49.227535 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:49.227597 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:49.227648 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:49.227722 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1208 11:26:49.227800 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:49.227858 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:49.227924 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:49.227989 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:49.228055 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:49.228119 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:49.228195 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:49.228268 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:49.228332 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:49.228383 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:49.228417 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:49.228453 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:49.228485 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:49.228523 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:49.228558 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:49.228595 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:49.228632 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:49.228680 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:49.228713 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:49.228773 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:49.228808 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:49.228844 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:49.228885 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:49.228919 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:49.230481 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:49.230512 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:49.230551 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230602 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1208 11:26:49.230666 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230726 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230781 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230835 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230874 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.230920 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1208 11:26:49.230976 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.231026 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:49.231051 34645 natgateway.go:117] DescribeNatGateways: { } I1208 11:26:49.231097 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1208 11:26:49.231141 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W1208 11:26:49.231198 34645 api.go:53] PageSize not implemented I1208 11:26:49.231219 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:49.231245 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:49.231259 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1208 11:26:49.231312 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:49.231338 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:49.231354 34645 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I1208 11:26:49.231419 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.231449 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.231493 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:49.231528 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.231569 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:49.231625 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:49.231669 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1208 11:26:49.231707 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:49.231743 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I1208 11:26:49.232444 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:49.232556 34645 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I1208 11:26:49.232570 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I1208 11:26:49.232605 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-config:lt-2 ok I1208 11:26:49.232672 34645 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I1208 11:26:49.232744 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok iam-instance-profile:nodes.minimal-ipv6.example.com ok I1208 11:26:49.232762 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } volume:vol-1 ok I1208 11:26:49.232882 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:49.232923 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1208 11:26:49.233053 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I1208 11:26:49.232690 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I1208 11:26:49.232968 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:49.233132 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.233206 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W1208 11:26:49.233517 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:49.233542 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:49.233577 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.233632 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:49.233664 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1208 11:26:49.233721 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:49.233774 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1208 11:26:49.233903 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-1 ok I1208 11:26:49.233930 34645 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1208 11:26:49.233990 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:49.234071 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I1208 11:26:49.234124 34645 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1208 11:26:49.234140 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1208 11:26:49.234121 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I1208 11:26:49.234173 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } subnet:subnet-1 ok iam-role:masters.minimal-ipv6.example.com ok I1208 11:26:49.234199 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I1208 11:26:49.234244 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1208 11:26:49.234257 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1208 11:26:49.234220 34645 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I1208 11:26:49.234286 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } elastic-ip:eipalloc-1 ok iam-role:nodes.minimal-ipv6.example.com ok I1208 11:26:49.234382 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1208 11:26:49.234468 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:49.234551 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:49.234839 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.54s) === RUN TestLifecycleSharedVPC I1208 11:26:49.235118 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:49.235550 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:49.235601 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:49.235644 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:49.235690 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:49.235733 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:49.235761 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:49.235796 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:49.235822 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:49.235885 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I1208 11:26:49.260226 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:49.263285 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.263429 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.302602 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.403420 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.405839 34645 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1208 11:26:49.406246 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:49.406300 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.406709 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I1208 11:26:49.406813 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1208 11:26:49.406907 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:49.409943 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:49.410016 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I1208 11:26:49.410030 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.410139 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1208 11:26:49.410169 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:49.410276 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.410316 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:49.410472 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:49.410526 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:49.410560 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:49.410608 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.410773 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.411812 34645 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1208 11:26:49.412139 34645 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1208 11:26:49.412194 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1208 11:26:49.412203 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1208 11:26:49.412308 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1208 11:26:49.412518 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1208 11:26:49.412442 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:49.412663 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1208 11:26:49.412698 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1208 11:26:49.412959 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.413100 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:49.413252 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I1208 11:26:49.413195 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.413470 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:49.413594 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:49.413661 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.413531 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I1208 11:26:49.413713 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:49.413828 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:49.413889 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.413952 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:49.414004 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1208 11:26:49.414127 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:49.414182 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:49.414593 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1208 11:26:49.417809 34645 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1208 11:26:49.418020 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.418163 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.418339 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1208 11:26:49.418471 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.418636 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.418898 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1208 11:26:49.419049 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.419268 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.419409 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.419506 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.419612 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.419678 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.419923 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:49.419998 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1208 11:26:49.420069 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:49.420281 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.420510 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.420674 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1208 11:26:49.420743 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.420902 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.421065 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I1208 11:26:49.421221 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" }] }] } I1208 11:26:49.421353 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.421457 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.421553 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1208 11:26:49.424175 34645 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1208 11:26:49.424431 34645 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1208 11:26:49.424577 34645 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1208 11:26:49.424622 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.424660 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.424714 34645 dns.go:238] Pre-creating DNS records I1208 11:26:49.424744 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.431505 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:49.434582 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.434675 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.474808 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.582564 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.584708 34645 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1208 11:26:49.585119 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.585210 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:49.585283 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:49.585434 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:49.585567 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:49.585625 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:49.585704 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.585993 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] }] } I1208 11:26:49.588071 34645 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1208 11:26:49.588231 34645 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1208 11:26:49.588230 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1208 11:26:49.588436 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:49.588537 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1208 11:26:49.588571 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.588760 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.588916 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:49.588958 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1208 11:26:49.589122 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.589189 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.594356 34645 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1208 11:26:49.594539 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.594786 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.595038 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.595206 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.595496 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.595652 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.595797 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.598346 34645 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1208 11:26:49.598577 34645 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1208 11:26:49.598701 34645 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1208 11:26:49.601437 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:49.601498 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:49.601534 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:49.601565 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:49.601600 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:49.601632 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:49.601664 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:49.601696 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:49.601732 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:49.601768 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:49.601809 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:49.601842 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:49.601874 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:49.601909 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:49.601943 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:49.601979 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:49.602015 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:49.602052 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:49.602117 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:49.602152 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:49.602184 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:49.602220 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:49.602258 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:49.603647 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:49.603674 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:49.603715 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.603764 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1208 11:26:49.603817 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.603876 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.603918 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.603964 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.604006 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.604044 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1208 11:26:49.604092 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.604146 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1208 11:26:49.604183 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W1208 11:26:49.604237 34645 api.go:53] PageSize not implemented I1208 11:26:49.604259 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:49.604288 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:49.604309 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:49.604333 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:49.604351 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:49.604405 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.604433 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.604478 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:49.604517 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.604557 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:49.604595 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:49.604621 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.604998 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:49.605071 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.605065 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok autoscaling-group:nodes.sharedvpc.example.com ok I1208 11:26:49.605158 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I1208 11:26:49.605131 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I1208 11:26:49.605219 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1208 11:26:49.605288 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok W1208 11:26:49.605263 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:49.605336 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:49.605369 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I1208 11:26:49.605210 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } volume:vol-1 ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok I1208 11:26:49.605455 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.605517 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1208 11:26:49.605740 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:49.605762 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1208 11:26:49.605809 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1208 11:26:49.605913 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I1208 11:26:49.605989 34645 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I1208 11:26:49.605959 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:49.606015 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1208 11:26:49.606056 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I1208 11:26:49.606105 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I1208 11:26:49.606154 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1208 11:26:49.606166 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1208 11:26:49.606192 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I1208 11:26:49.606244 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:49.606527 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I1208 11:26:49.606798 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:49.607056 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:49.607108 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:49.607153 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:49.607204 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:49.607248 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:49.607284 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:49.607325 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:49.607353 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:49.607418 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I1208 11:26:49.641920 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:49.646282 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.646378 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.688661 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.885613 34645 iamrole.go:89] ListRoles: { } I1208 11:26:49.890494 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1208 11:26:49.890857 34645 api.go:53] PageSize not implemented I1208 11:26:49.890915 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:49.891078 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1208 11:26:49.891310 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:49.891484 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:49.891098 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I1208 11:26:49.891550 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.891558 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:49.891599 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I1208 11:26:49.891715 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.891942 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1208 11:26:49.891994 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:49.892098 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I1208 11:26:49.892225 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.892545 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:49.892589 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.892729 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.892944 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:49.893002 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:49.893113 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:49.893286 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:49.893352 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:49.893387 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:49.893416 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:49.893459 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.894308 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:49.894501 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.894550 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:49.894634 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.894705 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:49.894742 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:49.894862 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1208 11:26:49.894824 34645 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1208 11:26:49.894969 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1208 11:26:49.895046 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:49.894993 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:49.895055 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:49.895162 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1208 11:26:49.895238 34645 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I1208 11:26:49.895309 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I1208 11:26:49.895299 34645 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1208 11:26:49.895514 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.895550 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I1208 11:26:49.895631 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1208 11:26:49.895698 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:49.895743 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:49.895763 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1208 11:26:49.895857 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.895944 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.896036 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.896138 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:49.896208 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:49.896254 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.896350 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1208 11:26:49.896215 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1208 11:26:49.896458 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1208 11:26:49.896449 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1208 11:26:49.896574 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:49.896661 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:49.896703 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:49.896748 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I1208 11:26:49.896846 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:49.896948 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:49.896999 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:49.897040 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:49.897077 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:49.897222 34645 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I1208 11:26:49.897204 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:49.897330 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:49.897400 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:49.897450 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:49.897511 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:49.897571 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1208 11:26:49.897620 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:49.897654 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:49.897304 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1208 11:26:49.897701 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I1208 11:26:49.897829 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:49.897889 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:49.897945 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1208 11:26:49.898043 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:49.899854 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:49.900008 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.900116 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.900186 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:49.900294 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:49.900240 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:49.900404 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:49.900457 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1208 11:26:49.900416 34645 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" } ], Type: "network" } I1208 11:26:49.900547 34645 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1208 11:26:49.900507 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I1208 11:26:49.900635 34645 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1208 11:26:49.900704 34645 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:49.900770 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I1208 11:26:49.900980 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I1208 11:26:49.901216 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1208 11:26:49.901332 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1208 11:26:49.901546 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1208 11:26:49.901714 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.901953 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.902183 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.902412 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.902637 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I1208 11:26:49.902691 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1208 11:26:49.902903 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.903027 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:49.903169 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I1208 11:26:49.903245 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1208 11:26:49.903400 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.903572 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1208 11:26:49.903685 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I1208 11:26:49.903841 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1208 11:26:49.904008 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I1208 11:26:49.904105 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.904256 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" } ] }] } I1208 11:26:49.904405 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:49.904474 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1208 11:26:49.904631 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.904811 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.904890 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1208 11:26:49.904947 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I1208 11:26:49.905105 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I1208 11:26:49.905310 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1208 11:26:49.905534 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" } ] }] } I1208 11:26:49.905684 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.905847 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1208 11:26:49.905891 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I1208 11:26:49.905987 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.906143 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1208 11:26:49.906290 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1208 11:26:49.906331 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.906418 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1208 11:26:49.906572 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I1208 11:26:49.906611 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1208 11:26:49.906647 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.908290 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:49.908553 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:49.908827 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:49.908961 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:49.909104 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1208 11:26:49.909156 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:49.909197 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.909251 34645 dns.go:238] Pre-creating DNS records I1208 11:26:49.909277 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:49.922538 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:49.926931 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.927037 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:49.965923 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.164691 34645 iamrole.go:89] ListRoles: { } I1208 11:26:50.167441 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1208 11:26:50.167711 34645 api.go:53] PageSize not implemented I1208 11:26:50.167848 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.168035 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:50.168140 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:50.168170 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:50.168197 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:50.168304 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:50.168404 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:50.168386 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:50.168519 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.168718 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:50.168851 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.168988 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.170101 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:50.170286 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.170394 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:50.170449 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:50.170426 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:50.170454 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.170534 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.170646 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:50.170643 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:50.170698 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:50.170727 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.170737 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:50.170912 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:50.170977 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:50.171120 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:50.171172 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:50.171311 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:50.171466 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:50.171524 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.171689 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.171789 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.171875 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.172052 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:50.172135 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.172210 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.172276 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.175643 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:50.175793 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.175922 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.176019 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:50.176096 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:50.176126 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.176104 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.176182 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.176235 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.176276 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.176240 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.176312 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.176344 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.176459 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.176757 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.176963 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.177084 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.177156 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.177327 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.177467 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.180054 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:50.180139 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:50.180226 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:50.180251 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:50.180279 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.180357 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:50.180530 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I1208 11:26:50.186943 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.190573 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.202606 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:50.206786 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.206885 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.245389 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.443360 34645 iamrole.go:89] ListRoles: { } I1208 11:26:50.445767 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1208 11:26:50.445990 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1208 11:26:50.446115 34645 api.go:53] PageSize not implemented I1208 11:26:50.446164 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.446368 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:50.446386 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:50.446531 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:50.446692 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:50.446763 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:50.446831 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.446883 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.447026 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.447176 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:50.447282 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:50.449196 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:50.449361 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:50.449540 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:50.449526 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.449709 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:50.449711 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:50.449730 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.449811 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.449895 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:50.449907 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:50.449999 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.450097 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:50.450162 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:50.450198 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.450358 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:50.450413 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.450510 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.450594 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:50.450656 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.450743 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.450877 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:50.451028 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:50.451081 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.451182 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.451257 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:50.451359 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:50.454648 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:50.454814 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.454924 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.455123 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.455143 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:50.455228 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:50.455247 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.455288 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.455334 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.455374 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.455433 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.455468 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.455665 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I1208 11:26:50.455702 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.455732 34645 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I1208 11:26:50.455684 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.455762 34645 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I1208 11:26:50.455796 34645 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1208 11:26:50.455864 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.455895 34645 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1208 11:26:50.455960 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.455996 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.456104 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.456235 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.456344 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.456596 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.456723 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.456899 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.459653 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:50.459781 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:50.459880 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:50.459904 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:50.459930 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.460015 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:50.460178 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1208 11:26:50.460232 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.460273 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:50.460333 34645 dns.go:238] Pre-creating DNS records I1208 11:26:50.472218 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:50.477169 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.477336 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.517446 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.713044 34645 iamrole.go:89] ListRoles: { } I1208 11:26:50.714813 34645 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1208 11:26:50.715137 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.716577 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.716746 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:50.716696 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:50.716924 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:50.717000 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:50.717454 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1208 11:26:50.717715 34645 api.go:53] PageSize not implemented I1208 11:26:50.717662 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:50.717878 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:50.717973 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.718182 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.718559 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:50.723678 34645 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1208 11:26:50.723835 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:50.723933 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:50.724089 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.724183 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:50.724222 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:50.724298 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:50.724320 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.724359 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.724502 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:50.724494 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:50.724616 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.724578 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.724745 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:50.724801 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.725001 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.725298 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:50.725384 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.725606 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:50.725810 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:50.725945 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.726099 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.726258 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.726417 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:50.726476 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:50.726557 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:50.730464 34645 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1208 11:26:50.730732 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.730890 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I1208 11:26:50.730917 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1208 11:26:50.731006 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:50.731146 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.731166 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.731219 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.731261 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.731303 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:50.731337 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:50.731369 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:50.731370 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.731621 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:50.731724 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.731878 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.732193 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:50.732397 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.732602 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.732694 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.735627 34645 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1208 11:26:50.735769 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:50.735910 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:50.735954 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:50.736004 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:50.736135 34645 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1208 11:26:50.736283 34645 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I1208 11:26:50.740217 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:50.743932 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:50.756247 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:50.760478 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.760578 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:50.800314 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.000542 34645 iamrole.go:89] ListRoles: { } I1208 11:26:51.003029 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1208 11:26:51.003401 34645 api.go:53] PageSize not implemented I1208 11:26:51.003387 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1208 11:26:51.003537 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:51.003635 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:51.003733 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:51.003763 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:51.003996 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.004169 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.004335 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:51.004463 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:51.004494 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:51.004559 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.004604 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:51.004714 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.006415 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:51.006646 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:51.006570 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:51.006888 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:51.006818 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.006804 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:51.007016 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.007017 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.007103 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:51.007200 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:51.007202 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.007268 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.007450 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.007550 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.007694 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:51.007759 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:51.007947 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:51.007993 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.008069 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.008190 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.008268 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:51.008392 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I1208 11:26:51.008492 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.008613 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:51.008703 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:51.008792 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:51.012238 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:51.012402 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.012548 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.012593 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:51.012685 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:51.012706 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.012756 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.012766 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.012821 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.012937 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.012927 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.012996 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.013044 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.013119 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.013469 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.013873 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.015018 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.015687 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.016043 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.016251 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.020880 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.020980 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.021173 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:51.021382 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:51.021494 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:51.021523 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:51.021550 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.021645 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:51.021818 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:51.021873 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.021910 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:51.021969 34645 dns.go:238] Pre-creating DNS records I1208 11:26:51.030786 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:51.035524 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.035631 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.074486 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.274995 34645 iamrole.go:89] ListRoles: { } I1208 11:26:51.277634 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1208 11:26:51.277855 34645 api.go:53] PageSize not implemented I1208 11:26:51.277986 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:51.278090 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:51.278236 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.278426 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:51.278487 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.278631 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:51.278796 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1208 11:26:51.278909 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:51.279013 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.279134 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:51.279166 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:51.279227 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.279533 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.280430 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:51.280576 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.280713 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:51.280694 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.280801 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:51.280837 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.280891 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:51.280977 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:51.280989 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.280922 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:51.281066 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.281102 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:51.281164 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.281340 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:51.281392 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.281529 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:51.281586 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.281838 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.281944 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:51.282001 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:51.282178 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:51.282403 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:51.282475 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.282668 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.282926 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:51.283221 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:51.286107 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:51.286274 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.286493 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.286509 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:51.286620 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:51.286592 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.286646 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.286719 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.286765 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.286774 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.286835 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.286877 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.286919 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.286923 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.287175 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.287721 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.287938 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.288165 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.288442 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.288622 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.289542 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:51.289670 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:51.289776 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:51.289819 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:51.289865 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.289999 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:51.290126 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I1208 11:26:51.295486 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.299131 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.299243 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.308807 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:51.313181 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.313308 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.313402 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.353612 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.556516 34645 iamrole.go:89] ListRoles: { } I1208 11:26:51.559091 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1208 11:26:51.559566 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:51.559551 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.559758 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:51.559826 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W1208 11:26:51.559869 34645 api.go:53] PageSize not implemented I1208 11:26:51.559919 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.560202 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.560417 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:51.560601 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.560675 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:51.560828 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:51.560983 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1208 11:26:51.561139 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:51.561292 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.562270 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:51.562438 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:51.562458 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1208 11:26:51.562565 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.562673 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:51.562650 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.562751 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:51.562861 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.562813 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.562882 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:51.563019 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.563161 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.563265 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:51.563436 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:51.563509 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.563656 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:51.563702 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:51.563917 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:51.563982 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I1208 11:26:51.564155 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:51.564200 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.564293 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:51.564461 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:51.564504 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1208 11:26:51.564612 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.564703 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:51.568044 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:51.568223 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.568312 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:51.568419 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:51.568412 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.568448 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.568570 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.568630 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.568671 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.568643 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.568705 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.568754 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.568814 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.569006 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.569244 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.569550 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.569704 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.569793 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.569864 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.569965 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.576593 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:51.576693 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:51.576790 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:51.576813 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:51.576836 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.577028 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:51.577222 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:51.577280 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.577320 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:51.577383 34645 dns.go:238] Pre-creating DNS records I1208 11:26:51.585431 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:51.590583 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.590688 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.590749 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.631336 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.826239 34645 iamrole.go:89] ListRoles: { } I1208 11:26:51.827971 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1208 11:26:51.828122 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.828301 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1208 11:26:51.828484 34645 api.go:53] PageSize not implemented I1208 11:26:51.830711 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:51.830933 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:51.831024 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:51.831100 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:51.831175 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1208 11:26:51.831250 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.831389 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:51.831418 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:51.831473 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.831567 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.828534 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:51.835242 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:51.835365 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:51.835514 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:51.835461 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.835634 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.835602 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:51.835771 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.835771 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:51.835952 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:51.835879 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:51.836039 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.836098 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.836193 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.836294 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.836385 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.836559 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:51.836613 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:51.836771 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:51.836836 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.836971 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:51.837137 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:51.837191 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.837451 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:51.837499 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:51.837616 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:51.837835 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:51.841144 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:51.841310 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.841457 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:51.841551 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:51.841572 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.841634 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.841672 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.841711 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:51.841748 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:51.841689 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.841780 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:51.841848 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.842132 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.842571 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.842805 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:51.842947 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.843138 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.843318 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.843478 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.843639 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:51.845266 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:51.845379 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:51.845493 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:51.845518 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:51.845543 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:51.845634 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:51.845798 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I1208 11:26:51.849633 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:51.853287 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.853385 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.853536 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.863388 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:51.867673 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.867776 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.867838 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.867897 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:51.906653 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.111698 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.114180 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1208 11:26:52.114370 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:52.114461 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:52.114650 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:52.114760 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W1208 11:26:52.115008 34645 api.go:53] PageSize not implemented I1208 11:26:52.114933 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.115103 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:52.115154 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:52.115212 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:52.115385 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1208 11:26:52.115616 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.115795 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.115964 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:52.116108 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.117730 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:52.117872 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.118072 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:52.118081 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:52.118088 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.118180 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:52.118195 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:52.118262 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:52.118220 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:52.118330 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:52.118410 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.118473 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:52.118537 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.118697 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:52.118760 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.118860 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:52.119033 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.119088 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:52.119233 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:52.119310 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.119462 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.119595 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.119739 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:52.119793 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.119954 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.120100 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.123402 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:52.123577 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:52.123663 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:52.123685 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.123725 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.123760 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:52.123802 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:52.123838 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:52.123870 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:52.123592 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.124262 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.124528 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.124706 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.124944 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.125165 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.125330 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.125475 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.125573 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.125723 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.125831 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.127421 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:52.127550 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:52.127661 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:52.127709 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:52.127758 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.127879 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:52.128033 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:52.128088 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.128126 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.128183 34645 dns.go:238] Pre-creating DNS records I1208 11:26:52.140663 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1208 11:26:52.145467 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.145579 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.145649 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.145708 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.184743 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.380231 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.381867 34645 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1208 11:26:52.382157 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1208 11:26:52.382283 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1208 11:26:52.384743 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1208 11:26:52.384963 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1208 11:26:52.385009 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:52.385214 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1208 11:26:52.385486 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1208 11:26:52.382414 34645 api.go:53] PageSize not implemented I1208 11:26:52.385734 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.386043 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:52.386121 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.386888 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.387010 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:52.387094 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.389267 34645 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1208 11:26:52.389404 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.389494 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1208 11:26:52.389594 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:52.389589 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.389621 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:52.389744 34645 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:52.389693 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.389762 34645 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1208 11:26:52.389840 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:52.389881 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.390021 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I1208 11:26:52.390210 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1208 11:26:52.390427 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:52.390523 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.390667 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.390813 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.390982 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:52.391026 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1208 11:26:52.391146 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.391245 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.391349 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1208 11:26:52.391471 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.391598 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.391677 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:52.391722 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1208 11:26:52.396776 34645 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1208 11:26:52.397058 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.397235 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.397242 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:52.397443 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:52.397464 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.397510 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.397431 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.397545 34645 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:52.397690 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I1208 11:26:52.397733 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I1208 11:26:52.397700 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.397765 34645 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1208 11:26:52.397807 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.398034 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.398387 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.398528 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.398679 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.398848 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.398990 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:52.400863 34645 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1208 11:26:52.400949 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:52.401035 34645 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1208 11:26:52.401059 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1208 11:26:52.401081 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.401153 34645 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1208 11:26:52.401306 34645 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1208 11:26:52.401766 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:52.401840 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:52.401878 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:52.401910 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:52.401945 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:52.401986 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:52.402021 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:52.402054 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:52.402118 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:52.402157 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:52.402214 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:52.402260 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:52.402295 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:52.402328 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:52.402367 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:52.402400 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:52.402438 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:52.402485 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:52.402524 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:52.402562 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:52.402600 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:52.402635 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:52.402678 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1208 11:26:52.402720 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:52.402753 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:52.402789 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:52.402827 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:52.402865 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:52.402903 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:52.405725 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:52.405760 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:52.405798 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.405861 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1208 11:26:52.405928 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.405991 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406044 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406107 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406150 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406200 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1208 11:26:52.406260 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406336 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1208 11:26:52.406379 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W1208 11:26:52.406438 34645 api.go:53] PageSize not implemented I1208 11:26:52.406461 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:52.406489 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:52.406504 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1208 11:26:52.406546 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:52.406569 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:52.406586 34645 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I1208 11:26:52.406661 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.406691 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.406736 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:52.406775 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.406818 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:52.406868 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:52.406915 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1208 11:26:52.407533 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1208 11:26:52.407566 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1208 11:26:52.407494 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok I1208 11:26:52.407684 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } autoscaling-group:nodes.complex.example.com ok I1208 11:26:52.407627 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I1208 11:26:52.407643 34645 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I1208 11:26:52.407746 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I1208 11:26:52.407828 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I1208 11:26:52.407842 34645 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I1208 11:26:52.407922 34645 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I1208 11:26:52.407762 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W1208 11:26:52.408129 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:52.408153 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:52.408188 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I1208 11:26:52.408306 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:52.408412 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W1208 11:26:52.408661 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:52.408697 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:52.408775 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1208 11:26:52.408877 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.408947 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:52.408982 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:52.409039 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1208 11:26:52.409099 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:52.409193 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:52.409215 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I1208 11:26:52.409247 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1208 11:26:52.409311 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I1208 11:26:52.409351 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1208 11:26:52.409401 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I1208 11:26:52.409435 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I1208 11:26:52.409482 34645 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I1208 11:26:52.409495 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1208 11:26:52.409521 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I1208 11:26:52.409582 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1208 11:26:52.409662 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:52.409744 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:52.410036 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.80s) === RUN TestLifecycleExternalLB I1208 11:26:52.410348 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:52.410614 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:52.410660 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:52.410703 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:52.410750 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:52.410789 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:52.410817 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:52.410852 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:52.410878 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:52.410943 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I1208 11:26:52.435973 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:52.439126 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.439229 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.478070 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.583105 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.584905 34645 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1208 11:26:52.585116 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.585330 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:52.585417 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:52.585499 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:52.585532 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:52.585557 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.585525 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:52.585614 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W1208 11:26:52.585817 34645 api.go:53] PageSize not implemented W1208 11:26:52.585877 34645 api.go:53] PageSize not implemented I1208 11:26:52.585804 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.585598 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:52.585992 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1208 11:26:52.586029 34645 api.go:53] PageSize not implemented I1208 11:26:52.585978 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I1208 11:26:52.586084 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.586049 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:52.586125 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } I1208 11:26:52.586249 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:52.586262 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:52.586305 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.586378 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I1208 11:26:52.586449 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1208 11:26:52.586348 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:52.586481 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.586546 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:52.586664 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.586840 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1208 11:26:52.586882 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:52.587195 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:52.587310 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:52.587378 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:52.587428 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:52.587505 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.587993 34645 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1208 11:26:52.588116 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1208 11:26:52.588243 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:52.588224 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1208 11:26:52.588343 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.588371 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:52.588381 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1208 11:26:52.588445 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.588623 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1208 11:26:52.588758 34645 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1208 11:26:52.588672 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1208 11:26:52.588695 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1208 11:26:52.588949 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:52.589064 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:52.589120 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.589164 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.589161 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1208 11:26:52.589345 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.589538 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1208 11:26:52.589771 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:52.589881 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:52.589946 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.590030 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:52.590078 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:52.590114 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:52.590162 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.590208 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:52.590266 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1208 11:26:52.590344 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:52.590391 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:52.590913 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1208 11:26:52.595620 34645 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1208 11:26:52.596649 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.596774 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.596959 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:52.597020 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:52.597232 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.597391 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1208 11:26:52.597568 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.598224 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.598392 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.598537 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.598606 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.598683 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.598889 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.599024 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.599151 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.599288 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.599416 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1208 11:26:52.599544 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:52.599599 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.599770 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:52.599856 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.600037 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.600170 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1208 11:26:52.600504 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I1208 11:26:52.600720 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1208 11:26:52.602643 34645 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1208 11:26:52.602977 34645 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1208 11:26:52.603273 34645 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1208 11:26:52.603355 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.603415 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.603502 34645 dns.go:238] Pre-creating DNS records I1208 11:26:52.603553 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.610101 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:52.613213 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.613314 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.651885 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.755384 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.756930 34645 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1208 11:26:52.757080 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1208 11:26:52.757214 34645 api.go:53] PageSize not implemented I1208 11:26:52.757426 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:52.759151 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:52.759235 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1208 11:26:52.759419 34645 api.go:53] PageSize not implemented I1208 11:26:52.759447 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1208 11:26:52.759576 34645 api.go:53] PageSize not implemented I1208 11:26:52.759637 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:52.759771 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1208 11:26:52.759963 34645 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:52.760108 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:52.760389 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.760550 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:52.760593 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:52.760641 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.760708 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.760956 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.761036 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.761397 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.761604 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.764724 34645 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1208 11:26:52.764886 34645 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1208 11:26:52.765038 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1208 11:26:52.764951 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1208 11:26:52.765134 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1208 11:26:52.765356 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.765426 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.765619 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:52.765666 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.765766 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.765902 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.766026 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:52.766120 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.770401 34645 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1208 11:26:52.770614 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.770949 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.771149 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.771381 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.771677 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.771847 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.772073 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.775187 34645 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1208 11:26:52.775707 34645 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1208 11:26:52.775889 34645 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1208 11:26:52.779222 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:52.779285 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:52.779320 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:52.779354 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:52.779387 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:52.779424 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:52.779466 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:52.779502 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:52.779535 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:52.779572 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:52.779605 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:52.779644 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:52.779685 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:52.779716 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:52.779748 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:52.779783 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:52.779814 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:52.779848 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:52.779883 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:52.779922 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:52.779954 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:52.779986 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:52.780016 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:52.780050 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:52.780088 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:52.780122 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:52.781532 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:52.781559 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:52.781598 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781649 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1208 11:26:52.781702 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781757 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781804 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781857 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781898 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.781937 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1208 11:26:52.781982 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.782036 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1208 11:26:52.782095 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W1208 11:26:52.782153 34645 api.go:53] PageSize not implemented I1208 11:26:52.782176 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:52.782202 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:52.782219 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:52.782244 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:52.782260 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:52.782314 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.782342 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.782392 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:52.782429 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.782472 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:52.782515 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:52.782550 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:52.782997 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1208 11:26:52.783070 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:52.783128 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I1208 11:26:52.783116 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } autoscaling-group:nodes.externallb.example.com ok I1208 11:26:52.783177 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.783209 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I1208 11:26:52.783073 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.externallb.example.com. ok I1208 11:26:52.783256 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1208 11:26:52.783353 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I1208 11:26:52.783231 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1208 11:26:52.783448 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:52.783472 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I1208 11:26:52.783516 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1208 11:26:52.783581 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:52.783618 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.783664 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1208 11:26:52.783773 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:52.783786 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:52.783850 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I1208 11:26:52.783904 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I1208 11:26:52.783920 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1208 11:26:52.783902 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I1208 11:26:52.783952 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } subnet:subnet-1 ok iam-role:nodes.externallb.example.com ok I1208 11:26:52.783980 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I1208 11:26:52.784021 34645 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I1208 11:26:52.784037 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1208 11:26:52.784063 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I1208 11:26:52.784116 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:52.784185 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:52.784252 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:52.784509 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I1208 11:26:52.784758 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:52.785031 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:52.785077 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:52.785118 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:52.785165 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:52.785206 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:52.785237 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:52.785273 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:52.785300 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:52.785364 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I1208 11:26:52.809626 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:52.812695 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.812790 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.852218 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.954730 34645 iamrole.go:89] ListRoles: { } I1208 11:26:52.956442 34645 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1208 11:26:52.956649 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:52.956864 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:52.957005 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:52.956982 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:52.957212 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:52.957236 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I1208 11:26:52.957311 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:52.957435 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:52.957396 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1208 11:26:52.957531 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:52.957642 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I1208 11:26:52.957776 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.957752 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I1208 11:26:52.957861 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:52.958069 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:52.958320 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:52.958397 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.959462 34645 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1208 11:26:52.959582 34645 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1208 11:26:52.959629 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1208 11:26:52.959631 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1208 11:26:52.959749 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1208 11:26:52.959791 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1208 11:26:52.959915 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1208 11:26:52.959903 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:52.959973 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:52.960095 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:52.960143 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:52.960188 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:52.960358 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1208 11:26:52.960442 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1208 11:26:52.960610 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:52.960729 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:52.960791 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:52.960837 34645 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I1208 11:26:52.961900 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1208 11:26:52.964620 34645 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1208 11:26:52.964843 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.965026 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" }] }] } I1208 11:26:52.965181 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.965323 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.965574 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.965685 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.965845 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.965995 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.966131 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.966332 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.966521 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.966668 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.966809 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.967003 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.967157 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.967301 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.967443 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1208 11:26:52.967576 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1208 11:26:52.968325 34645 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1208 11:26:52.968594 34645 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1208 11:26:52.968741 34645 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1208 11:26:52.968791 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:52.968826 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.968880 34645 dns.go:238] Pre-creating DNS records I1208 11:26:52.968909 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:52.978257 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:52.982292 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:52.982393 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.021713 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.125528 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.126992 34645 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1208 11:26:53.127243 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] }] } I1208 11:26:53.129099 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:53.127875 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.129374 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.129623 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.129819 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:53.129925 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:53.129962 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:53.133616 34645 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1208 11:26:53.133744 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1208 11:26:53.134451 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1208 11:26:53.134627 34645 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1208 11:26:53.134628 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1208 11:26:53.134808 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.134864 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1208 11:26:53.135040 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.135121 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:53.139978 34645 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1208 11:26:53.140279 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.140513 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.140868 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.141050 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.143169 34645 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1208 11:26:53.143401 34645 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1208 11:26:53.143563 34645 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1208 11:26:53.146497 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:53.146564 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:53.146601 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:53.146633 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:53.146667 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.146701 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:53.146731 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:53.146766 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:53.146801 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:53.146838 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:53.146877 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:53.146910 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:53.146939 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:53.146967 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:53.146999 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:53.147036 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:53.147074 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:53.147107 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:53.147140 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:53.147177 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:53.147212 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:53.148649 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:53.148680 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:53.148719 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.148790 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1208 11:26:53.148865 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.148929 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.148971 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.149012 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.149057 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.149095 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1208 11:26:53.149133 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.149176 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1208 11:26:53.149211 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W1208 11:26:53.149262 34645 api.go:53] PageSize not implemented I1208 11:26:53.149284 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:53.149313 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:53.149330 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:53.149356 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:53.149375 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:53.149425 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.149453 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.149503 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:53.149541 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.149591 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:53.149633 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:53.149999 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:53.150076 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I1208 11:26:53.150063 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } volume:vol-1 ok I1208 11:26:53.150125 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.150156 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok autoscaling-config:lt-2 ok I1208 11:26:53.150197 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1208 11:26:53.150255 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok I1208 11:26:53.150207 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1208 11:26:53.150348 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.150371 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:53.150406 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.150253 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I1208 11:26:53.150467 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1208 11:26:53.150682 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.150703 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1208 11:26:53.150752 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1208 11:26:53.150852 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I1208 11:26:53.150910 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1208 11:26:53.150926 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1208 11:26:53.150967 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I1208 11:26:53.151000 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I1208 11:26:53.151042 34645 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1208 11:26:53.151055 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1208 11:26:53.151083 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I1208 11:26:53.151341 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I1208 11:26:53.151618 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:53.151910 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:53.151960 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:53.152005 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:53.152054 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:53.152123 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:53.152154 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:53.152191 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:53.152221 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:53.152284 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I1208 11:26:53.180949 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:53.184257 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.184347 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.184414 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.223451 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.331346 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.333709 34645 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1208 11:26:53.334212 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.334287 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.334422 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:53.334412 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1208 11:26:53.334525 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:53.334596 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I1208 11:26:53.334695 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:53.334758 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.334798 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.334937 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I1208 11:26:53.334999 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:53.335185 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:53.335211 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.335247 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.335338 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1208 11:26:53.335462 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.335658 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:53.335723 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:53.335841 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:53.337206 34645 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1208 11:26:53.337339 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1208 11:26:53.337531 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1208 11:26:53.337591 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1208 11:26:53.337573 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1208 11:26:53.337625 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.337792 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1208 11:26:53.337743 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1208 11:26:53.337974 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.338012 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.338176 34645 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1208 11:26:53.338204 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:53.338297 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.338390 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1208 11:26:53.338361 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1208 11:26:53.338699 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:53.338802 34645 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] } I1208 11:26:53.338853 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.338894 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1208 11:26:53.339109 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1208 11:26:53.339287 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1208 11:26:53.339483 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.339575 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:53.339682 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.339761 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.339844 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.339933 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:53.339981 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.340017 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:53.340059 34645 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] } I1208 11:26:53.340141 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:53.340183 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.340217 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:53.340257 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.340289 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:53.340328 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:53.340470 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.342542 34645 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1208 11:26:53.342675 34645 api.go:53] PageSize not implemented I1208 11:26:53.342758 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1208 11:26:53.342804 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1208 11:26:53.343000 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.344137 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:53.342958 34645 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1208 11:26:53.344403 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } W1208 11:26:53.344485 34645 api.go:53] PageSize not implemented I1208 11:26:53.344520 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1208 11:26:53.344336 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.344846 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.344899 34645 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1208 11:26:53.345004 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.344871 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I1208 11:26:53.345043 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1208 11:26:53.345130 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1208 11:26:53.345152 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.345332 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1208 11:26:53.345537 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1208 11:26:53.345473 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" }] }] } I1208 11:26:53.345822 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.346198 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I1208 11:26:53.346495 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:53.346751 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.347088 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.347991 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.348162 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.348261 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.348354 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.348512 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1208 11:26:53.348649 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:53.349622 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1208 11:26:53.349749 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.349901 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.349982 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.350126 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.350265 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.350408 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.350537 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1208 11:26:53.350671 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1208 11:26:53.350808 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.350853 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.350894 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.353322 34645 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1208 11:26:53.353547 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.353716 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.353857 34645 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1208 11:26:53.354002 34645 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1208 11:26:53.354059 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.354089 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.354138 34645 dns.go:238] Pre-creating DNS records I1208 11:26:53.354165 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.361353 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:53.364658 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.364752 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.364819 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.403012 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.504834 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.506677 34645 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1208 11:26:53.507055 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.510092 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I1208 11:26:53.510346 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:53.510496 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:53.510656 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:53.510709 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:53.510779 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.511014 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.514062 34645 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1208 11:26:53.514269 34645 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1208 11:26:53.514194 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1208 11:26:53.514358 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1208 11:26:53.514416 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1208 11:26:53.514484 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1208 11:26:53.514599 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1208 11:26:53.514693 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:53.514868 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1208 11:26:53.515205 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.515286 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1208 11:26:53.515528 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.515599 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1208 11:26:53.515767 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.515825 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1208 11:26:53.515973 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.516036 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:53.519452 34645 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1208 11:26:53.519605 34645 api.go:53] PageSize not implemented I1208 11:26:53.519633 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } W1208 11:26:53.519737 34645 api.go:53] PageSize not implemented I1208 11:26:53.519787 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.519946 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.520034 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1208 11:26:53.520181 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1208 11:26:53.520076 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.520297 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1208 11:26:53.520414 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.520713 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.520967 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.521177 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.521409 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.524180 34645 executor.go:111] Tasks: 100 done / 107 total; 4 can run W1208 11:26:53.524325 34645 api.go:53] PageSize not implemented I1208 11:26:53.524288 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.524357 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.524417 34645 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I1208 11:26:53.524509 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W1208 11:26:53.524594 34645 api.go:53] PageSize not implemented I1208 11:26:53.524620 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.524857 34645 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1208 11:26:53.525040 34645 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1208 11:26:53.528757 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:53.528823 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:53.528863 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:53.528896 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:53.528929 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.528962 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:53.528992 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:53.529025 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:53.529062 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:53.529100 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:53.529137 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1208 11:26:53.529181 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:53.529212 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:53.529244 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:53.529275 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:53.529310 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:53.529344 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:53.529378 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:53.529412 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:53.529449 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:53.529490 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:53.529526 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:53.529557 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:53.529592 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:53.529627 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:53.531242 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:53.531271 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:53.531313 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531377 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1208 11:26:53.531454 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531511 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531554 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531595 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531636 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531671 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1208 11:26:53.531710 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531752 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1208 11:26:53.531785 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W1208 11:26:53.531829 34645 api.go:53] PageSize not implemented I1208 11:26:53.531846 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1208 11:26:53.531896 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:53.531926 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:53.531945 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:53.531969 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:53.531988 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:53.532038 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.532086 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.532135 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:53.532177 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.532220 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:53.532262 34645 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok autoscaling-group:bastion.private-shared-subnet.example.com ok I1208 11:26:53.532834 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.532845 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I1208 11:26:53.532935 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I1208 11:26:53.532852 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1208 11:26:53.532984 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } autoscaling-group:nodes.private-shared-subnet.example.com ok load-balancer:bastion-private-shared-su-5ol32q ok I1208 11:26:53.532993 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.533076 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.533034 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } iam-instance-profile:masters.private-shared-subnet.example.com ok I1208 11:26:53.533146 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1208 11:26:53.533229 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I1208 11:26:53.532957 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok I1208 11:26:53.533163 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1208 11:26:53.533347 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.533367 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I1208 11:26:53.533425 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1208 11:26:53.533489 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:53.533550 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1208 11:26:53.533672 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.533692 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1208 11:26:53.533724 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.533777 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W1208 11:26:53.533978 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.534001 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:53.534033 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I1208 11:26:53.534140 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.534158 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.534223 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1208 11:26:53.534240 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1208 11:26:53.534287 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I1208 11:26:53.534247 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1208 11:26:53.534340 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.534360 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } I1208 11:26:53.534331 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } security-group:sg-5 ok I1208 11:26:53.534410 34645 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1208 11:26:53.534405 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.534442 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1208 11:26:53.534482 34645 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I1208 11:26:53.534460 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1208 11:26:53.534563 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.534531 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.534578 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:53.534594 34645 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1208 11:26:53.534608 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1208 11:26:53.534639 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I1208 11:26:53.534970 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I1208 11:26:53.535227 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:53.535529 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:53.535579 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:53.535622 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:53.535669 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:53.535710 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:53.535739 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:53.535778 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:53.535808 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:53.535871 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I1208 11:26:53.568461 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:53.572291 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.572386 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.572451 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.612142 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.723272 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.725662 34645 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1208 11:26:53.725865 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I1208 11:26:53.726094 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:53.726298 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:53.726400 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.726500 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.726523 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1208 11:26:53.726578 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.726665 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I1208 11:26:53.726713 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.726783 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:53.726839 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1208 11:26:53.726976 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:53.727100 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:53.727141 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:53.727120 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.727179 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" }] }] } I1208 11:26:53.727250 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I1208 11:26:53.727344 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:53.727458 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:53.729067 34645 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1208 11:26:53.729211 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1208 11:26:53.729375 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.729426 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1208 11:26:53.729596 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.729599 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.729720 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1208 11:26:53.729750 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.729794 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.729832 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I1208 11:26:53.729915 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:53.729955 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1208 11:26:53.730016 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1208 11:26:53.729977 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I1208 11:26:53.730106 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.730186 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1208 11:26:53.730223 34645 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1208 11:26:53.730211 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:53.730259 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.730308 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1208 11:26:53.730442 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.730593 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.730710 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:53.730644 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.730805 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.730903 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1208 11:26:53.731017 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.731106 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I1208 11:26:53.731210 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1208 11:26:53.731318 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:53.731359 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:53.731411 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:53.731478 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.731313 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I1208 11:26:53.731560 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:53.731630 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:53.731692 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:53.731753 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.731381 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.731794 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:53.731948 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I1208 11:26:53.731961 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.732105 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:53.732169 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:53.732208 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1208 11:26:53.732299 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:53.732342 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.732386 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:53.732429 34645 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I1208 11:26:53.734594 34645 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1208 11:26:53.734739 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.734846 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } W1208 11:26:53.734988 34645 api.go:53] PageSize not implemented I1208 11:26:53.735101 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.735187 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1208 11:26:53.735309 34645 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] } I1208 11:26:53.735425 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1208 11:26:53.735378 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:53.735480 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1208 11:26:53.735561 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } W1208 11:26:53.735682 34645 api.go:53] PageSize not implemented I1208 11:26:53.735624 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1208 11:26:53.735708 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1208 11:26:53.735836 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I1208 11:26:53.735936 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.735993 34645 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1208 11:26:53.736082 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.736026 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.736195 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I1208 11:26:53.736117 34645 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1208 11:26:53.736261 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.736307 34645 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1208 11:26:53.736347 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.736491 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.736564 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1208 11:26:53.736652 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.736748 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.736905 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.737054 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:53.737191 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1208 11:26:53.737328 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.737461 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.737584 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1208 11:26:53.737686 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I1208 11:26:53.737739 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.737880 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.738004 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:53.738134 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1208 11:26:53.738270 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I1208 11:26:53.738397 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1208 11:26:53.738531 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.738641 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.738703 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.738911 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.739085 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.739221 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.739296 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:53.739359 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.739478 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1208 11:26:53.739543 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1208 11:26:53.739712 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.739823 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1208 11:26:53.740000 34645 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I1208 11:26:53.740058 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1208 11:26:53.740122 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.740232 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1208 11:26:53.740303 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.742618 34645 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1208 11:26:53.742733 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.742844 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.742901 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.742997 34645 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:53.743161 34645 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1208 11:26:53.743256 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:53.743321 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:53.743364 34645 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1208 11:26:53.743460 34645 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1208 11:26:53.743558 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.743730 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1208 11:26:53.743802 34645 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1208 11:26:53.743843 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.743875 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.743924 34645 dns.go:238] Pre-creating DNS records I1208 11:26:53.743949 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.750238 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1208 11:26:53.753568 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.753665 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.753735 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.792779 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.904077 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.906454 34645 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1208 11:26:53.906712 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.906877 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.906940 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:53.907131 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:53.907229 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.907430 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1208 11:26:53.907507 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1208 11:26:53.907536 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1208 11:26:53.910033 34645 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1208 11:26:53.910222 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1208 11:26:53.910307 34645 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1208 11:26:53.910192 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1208 11:26:53.910484 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.910643 34645 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1208 11:26:53.910714 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:53.910790 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.910872 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1208 11:26:53.911063 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.911122 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I1208 11:26:53.911322 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1208 11:26:53.911380 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1208 11:26:53.911744 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1208 11:26:53.911816 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.912018 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1208 11:26:53.912286 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1208 11:26:53.912471 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1208 11:26:53.912625 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.913689 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.913775 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.920186 34645 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1208 11:26:53.920392 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W1208 11:26:53.920545 34645 api.go:53] PageSize not implemented I1208 11:26:53.920588 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.920689 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.920740 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.920843 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.920862 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W1208 11:26:53.921009 34645 api.go:53] PageSize not implemented I1208 11:26:53.920988 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.921037 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.921139 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1208 11:26:53.921147 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.921176 34645 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1208 11:26:53.921376 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.921539 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1208 11:26:53.921674 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.921880 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.921966 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.922119 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.922217 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1208 11:26:53.924205 34645 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1208 11:26:53.924297 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.924328 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1208 11:26:53.924415 34645 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I1208 11:26:53.924377 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W1208 11:26:53.924436 34645 api.go:53] PageSize not implemented I1208 11:26:53.924468 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } W1208 11:26:53.924688 34645 api.go:53] PageSize not implemented I1208 11:26:53.924728 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1208 11:26:53.924940 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.925055 34645 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1208 11:26:53.925189 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:53.925430 34645 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1208 11:26:53.929580 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:53.929644 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:53.929683 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:53.929733 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:53.929774 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:53.929824 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:53.929862 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:53.929915 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:53.929968 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:53.930014 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:53.930070 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1208 11:26:53.930121 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1208 11:26:53.930164 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:53.930195 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:53.930234 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:53.930282 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:53.930334 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1208 11:26:53.930382 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:53.930429 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:53.930472 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1208 11:26:53.930522 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1208 11:26:53.930570 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1208 11:26:53.930618 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:53.930662 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:53.930714 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1208 11:26:53.930762 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:53.930809 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:53.930849 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:53.930896 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:53.930938 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:53.932636 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:53.932670 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:53.932726 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.932807 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1208 11:26:53.932907 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.932985 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933036 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933100 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933151 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933216 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1208 11:26:53.933284 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933357 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:53.933391 34645 natgateway.go:117] DescribeNatGateways: { } I1208 11:26:53.933446 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1208 11:26:53.933491 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W1208 11:26:53.933553 34645 api.go:53] PageSize not implemented I1208 11:26:53.933572 34645 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1208 11:26:53.933651 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:53.933683 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:53.933708 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:53.933746 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:53.933771 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1208 11:26:53.933835 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:53.933876 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:53.933928 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:53.933978 34645 iamrole.go:89] ListRoles: { } I1208 11:26:53.934043 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:53.934116 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:53.934155 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1208 11:26:53.934250 34645 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1208 11:26:53.934297 34645 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-config:lt-3 ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I1208 11:26:53.934918 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1208 11:26:53.934963 34645 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1208 11:26:53.934999 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok I1208 11:26:53.935079 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.935057 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } iam-instance-profile:bastions.private-shared-ip.example.com ok I1208 11:26:53.935109 34645 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok load-balancer:api-private-shared-ip-exa-ohatqj ok autoscaling-group:nodes.private-shared-ip.example.com ok I1208 11:26:53.935163 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.935026 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok I1208 11:26:53.935255 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I1208 11:26:53.935285 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.935334 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I1208 11:26:53.935137 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W1208 11:26:53.935441 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.935475 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1208 11:26:53.935528 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1208 11:26:53.935608 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1208 11:26:53.935680 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1208 11:26:53.935782 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.935805 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1208 11:26:53.935843 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1208 11:26:53.935918 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1208 11:26:53.936062 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.936093 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1208 11:26:53.936130 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1208 11:26:53.936212 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I1208 11:26:53.936350 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.936370 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:53.936422 34645 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1208 11:26:53.936467 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1208 11:26:53.936596 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.936616 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:53.936510 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.936657 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:53.936701 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } I1208 11:26:53.936715 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } iam-role:masters.private-shared-ip.example.com ok subnet:subnet-2 ok I1208 11:26:53.936758 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:53.936778 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.936846 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1208 11:26:53.936806 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1208 11:26:53.936883 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:53.936863 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1208 11:26:53.936898 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:53.936916 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I1208 11:26:53.936938 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.936983 34645 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1208 11:26:53.936997 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1208 11:26:53.937026 34645 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I1208 11:26:53.937102 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:53.937160 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1208 11:26:53.937472 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I1208 11:26:53.937818 34645 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:53.938092 34645 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:53.938140 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:53.938181 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:53.938228 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:53.938274 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:53.938310 34645 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:53.938351 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:53.938382 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:53.938447 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I1208 11:26:53.963035 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1208 11:26:53.966171 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:53.966262 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.005655 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:54.140742 34645 iamrole.go:89] ListRoles: { } I1208 11:26:54.142696 34645 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1208 11:26:54.143366 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.143487 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:54.143685 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.143695 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.143797 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1208 11:26:54.143942 34645 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.144011 34645 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] } I1208 11:26:54.143954 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:54.144136 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:54.144191 34645 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1208 11:26:54.144332 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:54.144386 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.144568 34645 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.144687 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:54.144733 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.144927 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:54.144906 34645 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.145014 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:54.145164 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1208 11:26:54.145243 34645 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1208 11:26:54.145283 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:54.145346 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.145414 34645 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.146204 34645 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1208 11:26:54.146515 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.146577 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.146692 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1208 11:26:54.146667 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1208 11:26:54.146787 34645 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.146871 34645 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1208 11:26:54.146773 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1208 11:26:54.146845 34645 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1208 11:26:54.147005 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.147210 34645 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:26:54.147280 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.147332 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.147523 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.147729 34645 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1208 11:26:54.148568 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.148704 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1208 11:26:54.149088 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1208 11:26:54.149313 34645 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I1208 11:26:54.149506 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:54.149650 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:54.149319 34645 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I1208 11:26:54.149822 34645 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] } I1208 11:26:54.150090 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:54.150230 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:54.150320 34645 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I1208 11:26:54.150567 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:54.150734 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:54.150905 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:54.151080 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:54.151146 34645 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.155827 34645 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1208 11:26:54.156156 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.156452 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1208 11:26:54.156606 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.156825 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1208 11:26:54.156971 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.157082 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.157332 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.157534 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" }] }] } I1208 11:26:54.157697 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.157780 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.157856 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.157941 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.158091 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.158345 34645 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:54.158403 34645 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1208 11:26:54.158447 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1208 11:26:54.158494 34645 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1208 11:26:54.158541 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.158696 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.158848 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.159043 34645 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.159206 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.159453 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.159641 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1208 11:26:54.159807 34645 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1208 11:26:54.160912 34645 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1208 11:26:54.161180 34645 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1208 11:26:54.161449 34645 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1208 11:26:54.161496 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:54.161533 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:54.161588 34645 dns.go:238] Pre-creating DNS records I1208 11:26:54.161620 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:54.168182 34645 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1208 11:26:54.172128 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.172226 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.210873 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:54.349410 34645 iamrole.go:89] ListRoles: { } I1208 11:26:54.351317 34645 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1208 11:26:54.351600 34645 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1208 11:26:54.351815 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.352024 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.352202 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.352449 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:54.352530 34645 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1208 11:26:54.352660 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.352768 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:26:54.352798 34645 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:26:54.352848 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.354803 34645 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1208 11:26:54.354938 34645 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1208 11:26:54.355120 34645 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1208 11:26:54.354976 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1208 11:26:54.355218 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.355511 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:54.355608 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1208 11:26:54.355845 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1208 11:26:54.355952 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.356080 34645 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:26:54.356197 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.356377 34645 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:26:54.356447 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.360078 34645 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1208 11:26:54.360420 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.360753 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.360926 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.361182 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.361318 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.361474 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1208 11:26:54.361620 34645 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1208 11:26:54.363724 34645 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1208 11:26:54.363963 34645 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1208 11:26:54.364117 34645 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1208 11:26:54.366988 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1208 11:26:54.367054 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1208 11:26:54.367089 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1208 11:26:54.367120 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1208 11:26:54.367155 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1208 11:26:54.367197 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1208 11:26:54.367231 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1208 11:26:54.367260 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1208 11:26:54.367293 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1208 11:26:54.367328 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1208 11:26:54.367364 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1208 11:26:54.367400 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1208 11:26:54.367444 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1208 11:26:54.367478 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1208 11:26:54.367510 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1208 11:26:54.367547 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1208 11:26:54.367581 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:26:54.367617 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1208 11:26:54.367653 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:26:54.367692 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1208 11:26:54.367726 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1208 11:26:54.367758 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1208 11:26:54.367791 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1208 11:26:54.367827 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1208 11:26:54.367863 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:26:54.367898 34645 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1208 11:26:54.369448 34645 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1208 11:26:54.369479 34645 keypairs.go:95] DescribeKeyPairs: { } I1208 11:26:54.369521 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369574 34645 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369634 34645 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369693 34645 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369746 34645 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369802 34645 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369845 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369885 34645 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369933 34645 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.369986 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1208 11:26:54.370031 34645 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W1208 11:26:54.370089 34645 api.go:53] PageSize not implemented I1208 11:26:54.370112 34645 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1208 11:26:54.370142 34645 loadbalancers.go:34] PageSize not implemented I1208 11:26:54.370160 34645 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1208 11:26:54.370185 34645 targetgroups.go:35] PageSize not implemented I1208 11:26:54.370202 34645 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1208 11:26:54.370254 34645 zones.go:93] ListHostedZonesPages { } I1208 11:26:54.370283 34645 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:54.370334 34645 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1208 11:26:54.370370 34645 iamrole.go:89] ListRoles: { } I1208 11:26:54.370415 34645 internetgateways.go:101] DescribeInternetGateways: { } I1208 11:26:54.370461 34645 routetable.go:53] DescribeRouteTables: { } I1208 11:26:54.370494 34645 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I1208 11:26:54.370955 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I1208 11:26:54.371040 34645 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1208 11:26:54.371006 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.371095 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I1208 11:26:54.371141 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I1208 11:26:54.371204 34645 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.371069 34645 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1208 11:26:54.371268 34645 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I1208 11:26:54.371163 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1208 11:26:54.371387 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:54.371410 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1208 11:26:54.371447 34645 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1208 11:26:54.371503 34645 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1208 11:26:54.371616 34645 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1208 11:26:54.371632 34645 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1208 11:26:54.371676 34645 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I1208 11:26:54.371792 34645 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1208 11:26:54.371858 34645 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1208 11:26:54.371937 34645 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1208 11:26:54.371955 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.372047 34645 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1208 11:26:54.372085 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1208 11:26:54.372140 34645 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I1208 11:26:54.372175 34645 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.372224 34645 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1208 11:26:54.372236 34645 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1208 11:26:54.372267 34645 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I1208 11:26:54.372331 34645 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1208 11:26:54.372414 34645 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1208 11:26:54.372487 34645 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1208 11:26:54.372760 34645 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.44s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 56.860s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.011s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I1208 11:25:35.786058 22607 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.077s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.045s === RUN TestPodController I1208 11:25:42.595120 30358 pod.go:59] starting pod controller I1208 11:25:42.595426 30358 pod.go:107] Got stop signal I1208 11:25:42.595485 30358 stoppable.go:62] shutting down controller I1208 11:25:42.595510 30358 pod.go:65] shutting down pod controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.059s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.065s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00018bde0} route53_test.go:93: Got zone list: [0xc00018be10] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00018bee0} route53_test.go:93: Got zone list: [0xc00018bf10] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00018bfb0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003b1800 0xc0001920a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e2110} route53_test.go:93: Got zone list: [0xc0003e2140] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc0001920c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e2240} route53_test.go:93: Got zone list: [0xc0003e2270] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc0001920e0} route53_test.go:187: Successfully added resource record set: {0xc0001f8600 0xc000192100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e24f0} route53_test.go:93: Got zone list: [0xc0003e2520] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192188} route53_test.go:199: Successfully added resource record set: {0xc0001f8780 0xc0001921a8} route53_test.go:118: Got 1 recordsets: [0xc0003e26c0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e27f0} route53_test.go:93: Got zone list: [0xc0003e2820] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192240} route53_test.go:221: Successfully added resource record set: {0xc0001f8900 0xc000192260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0001f8900 0xc000192260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e2b80} route53_test.go:93: Got zone list: [0xc0003e2bb0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192318} route53_test.go:246: Successfully removed resource set {0xc0001f8b00 0xc000192338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e2e30} route53_test.go:93: Got zone list: [0xc0003e2e60] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc0001923c0} route53_test.go:264: Successfully removed resource set {0xc0001f8c80 0xc0001923e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e3110} route53_test.go:93: Got zone list: [0xc0003e3140] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192470} commontests.go:192: Successfully added resource record set: {0xc0001f8e00 0xc000192490} commontests.go:45: Correctly replaced resource record {0xc0001f8e00 0xc000192490} -> {0xc0001f8f00 0xc0001924e0} commontests.go:142: Got 1 recordsets: [0xc0003e3450] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e3620} route53_test.go:93: Got zone list: [0xc0003e3650] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc000192590} commontests.go:192: Successfully added resource record set: {0xc0001f9180 0xc0001925b0} commontests.go:70: Correctly replaced resource record {0xc0001f9180 0xc0001925b0} -> {0xc0001f9280 0xc000192600} commontests.go:142: Got 1 recordsets: [0xc0003e3960] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003e3b80} route53_test.go:93: Got zone list: [0xc0003e3bb0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc0001926c0} commontests.go:192: Successfully added resource record set: {0xc0001f9500 0xc0001926e0} commontests.go:142: Got 2 recordsets: [0xc0003e3e20 0xc0003e3e40] commontests.go:142: Got 2 recordsets: [0xc0003e3eb0 0xc0003e3ed0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003f4070} route53_test.go:93: Got zone list: [0xc0003f40a0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003b1620 0xc0001927e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.056s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.034s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca0c0} clouddns_test.go:71: Got zone list: [0xc00068c318] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c2e8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca1c0} clouddns_test.go:71: Got zone list: [0xc00068c3c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c390} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca2c0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0000e2b00 0xc00068c468} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca380} clouddns_test.go:71: Got zone list: [0xc00068c510] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c4e0} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca4a0} clouddns_test.go:71: Got zone list: [0xc00068c630] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c600} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca5e0} clouddns_test.go:71: Got zone list: [0xc00068c780] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c750} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca740} clouddns_test.go:71: Got zone list: [0xc00068c978] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068c948} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca8a0} clouddns_test.go:71: Got zone list: [0xc00068caf8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068cac8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000ca9e0} clouddns_test.go:71: Got zone list: [0xc00068cc48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068cc18} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000cab40} clouddns_test.go:71: Got zone list: [0xc00068cdf8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068cdc8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000caca0} clouddns_test.go:71: Got zone list: [0xc00068d0b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068d080} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000cae20} clouddns_test.go:71: Got zone list: [0xc00068d3c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068d398} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682108 0xc0000cb000} clouddns_test.go:71: Got zone list: [0xc00068d788] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0000e2900 0xc00068d758} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.119s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.025s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.186s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I1208 11:25:56.808459 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.808785 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.808832 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.808878 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.808947 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.808998 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.809022 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.809064 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.809100 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.819297 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I1208 11:25:56.825879 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.826089 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.826130 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.826167 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.826205 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.826239 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.826261 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.826289 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.826309 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.827757 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I1208 11:25:56.831806 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.831982 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.832021 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.832078 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.832118 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.832171 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.832194 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.832220 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.832241 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.833582 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.835462 34552 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I1208 11:25:56.835485 34552 assetstore.go:111] bin/containerd I1208 11:25:56.835497 34552 assetstore.go:111] bin/containerd-shim I1208 11:25:56.835506 34552 assetstore.go:111] bin/containerd-shim-runc-v1 I1208 11:25:56.835515 34552 assetstore.go:111] bin/containerd-shim-runc-v2 I1208 11:25:56.835524 34552 assetstore.go:111] bin/containerd-stress I1208 11:25:56.835533 34552 assetstore.go:111] bin/ctr I1208 11:25:56.835581 34552 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I1208 11:25:56.835597 34552 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I1208 11:25:56.839322 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.839506 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.839549 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.839583 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.839625 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.839662 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.839683 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.839713 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.839735 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.841059 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.842709 34552 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I1208 11:25:56.843833 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.844006 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.844041 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.844070 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.844104 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.844181 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.844207 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.844237 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.844256 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.845495 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.847523 34552 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I1208 11:25:56.848335 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.848507 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.848541 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.848569 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.848603 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.848658 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.848682 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.848709 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.848729 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.850007 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.851676 34552 assetstore.go:108] Matching assets for "^docker/": I1208 11:25:56.851694 34552 assetstore.go:111] docker/containerd I1208 11:25:56.851704 34552 assetstore.go:111] docker/containerd-shim I1208 11:25:56.851715 34552 assetstore.go:111] docker/ctr I1208 11:25:56.851724 34552 assetstore.go:111] docker/docker I1208 11:25:56.851732 34552 assetstore.go:111] docker/docker-init I1208 11:25:56.851740 34552 assetstore.go:111] docker/docker-proxy I1208 11:25:56.851748 34552 assetstore.go:111] docker/dockerd I1208 11:25:56.851757 34552 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I1208 11:25:56.855421 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.855614 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.855647 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.855684 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.855719 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.855784 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.855811 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.855838 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.855858 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.858294 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.859682 34552 assetstore.go:108] Matching assets for "^docker/": I1208 11:25:56.859701 34552 assetstore.go:111] docker/containerd I1208 11:25:56.859709 34552 assetstore.go:111] docker/containerd-shim I1208 11:25:56.859715 34552 assetstore.go:111] docker/ctr I1208 11:25:56.859721 34552 assetstore.go:111] docker/docker I1208 11:25:56.859727 34552 assetstore.go:111] docker/docker-init I1208 11:25:56.859732 34552 assetstore.go:111] docker/docker-proxy I1208 11:25:56.859738 34552 assetstore.go:111] docker/dockerd I1208 11:25:56.859744 34552 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I1208 11:25:56.862382 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.862540 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.862570 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.862597 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.862625 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.862667 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.862681 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.862699 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.862712 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.863685 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.864987 34552 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I1208 11:25:56.865208 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.865356 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.865389 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.865425 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.865450 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.865509 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.865526 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.865544 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.865556 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.866395 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:56.867537 34552 assetstore.go:108] Matching assets for "^docker/": I1208 11:25:56.867551 34552 assetstore.go:111] docker/docker I1208 11:25:56.867558 34552 assetstore.go:111] docker/docker-containerd I1208 11:25:56.867565 34552 assetstore.go:111] docker/docker-containerd-ctr I1208 11:25:56.867571 34552 assetstore.go:111] docker/docker-containerd-shim I1208 11:25:56.867576 34552 assetstore.go:111] docker/docker-init I1208 11:25:56.867583 34552 assetstore.go:111] docker/docker-proxy I1208 11:25:56.867589 34552 assetstore.go:111] docker/docker-runc I1208 11:25:56.867594 34552 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I1208 11:25:56.870958 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.871114 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.871142 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.871164 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.871191 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.871213 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.871227 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.871245 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.871259 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.873209 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I1208 11:25:56.875132 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.875286 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.875311 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.875333 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.875357 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.875387 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.875415 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.875435 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.875448 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.877020 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I1208 11:25:56.879095 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.879272 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.879298 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.879319 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.879347 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.879368 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.879385 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.879405 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.879419 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.880822 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I1208 11:25:56.885664 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.885836 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.885873 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.885898 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.885922 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.885943 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.885956 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.885978 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.885992 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.888377 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I1208 11:25:56.901463 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.901686 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.901727 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.901771 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.901813 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.901846 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.901874 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.901908 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.901931 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.904536 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I1208 11:25:56.914147 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.914381 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.914423 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.914460 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.914502 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.914536 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.914561 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.914593 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.914615 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.916876 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I1208 11:25:56.926956 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.927168 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.927208 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.927245 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.927293 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.927326 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.927350 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.927381 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.927403 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.930009 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I1208 11:25:56.941817 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.942025 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:56.942080 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:56.942124 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:56.942167 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:56.942201 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:56.942223 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:56.942253 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:56.942275 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:56.944621 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.50s) === RUN TestKubeAPIServerBuilderARM64 I1208 11:25:58.440123 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.440293 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.440330 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.440359 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.440437 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.440465 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.440483 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.440508 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.440525 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.442240 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I1208 11:25:58.516247 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.516453 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.516498 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.516530 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.516565 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.516594 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.516611 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.516633 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.516651 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.518603 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I1208 11:25:58.524802 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.524967 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.525002 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.525028 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.525061 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.525088 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.525104 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.525129 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.525146 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.526886 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I1208 11:25:58.604655 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.604863 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.604897 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.604933 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.604970 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.605005 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.605030 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.605060 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.605080 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.607330 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I1208 11:25:58.682384 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.682632 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.682700 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.682745 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.682792 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.682832 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.682863 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.682898 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.682924 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.685606 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I1208 11:25:58.692567 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.692825 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.692872 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.692918 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.692964 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.693003 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.693032 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.693069 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.693097 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.695797 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I1208 11:25:58.766281 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.766474 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.766506 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.766533 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.766564 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.766589 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.766606 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.766629 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.766644 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.768252 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I1208 11:25:58.839938 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.840130 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.840162 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.840189 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.840220 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.840246 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.840263 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.840285 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.840301 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.841870 34552 zones.go:93] ListHostedZonesPages { } W1208 11:25:58.844244 34552 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I1208 11:25:58.846017 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.846183 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.846212 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.846238 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.846267 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.846290 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.846305 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.846325 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.846339 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.848262 34552 zones.go:93] ListHostedZonesPages { } W1208 11:25:58.917525 34552 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I1208 11:25:58.920404 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.920630 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.920680 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.920723 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.920773 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.920811 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.920839 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.920872 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.920895 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.923478 34552 zones.go:93] ListHostedZonesPages { } W1208 11:25:58.993749 34552 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I1208 11:25:58.995672 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:58.995903 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.995958 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:58.996015 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:58.996072 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:58.996115 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:58.996147 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:58.996185 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:58.996216 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:58.999173 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.002085 34552 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I1208 11:25:59.005247 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.005436 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.005499 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.005539 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.005585 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.005619 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.005653 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.005687 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.005712 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.008020 34552 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I1208 11:25:59.013772 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.013949 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.014005 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.014043 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.014087 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.014120 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.014152 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.014188 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.014213 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.016454 34552 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I1208 11:25:59.021972 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.022178 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.022217 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.022254 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.022299 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.022332 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.022363 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.022398 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.022422 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.025102 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.027740 34552 assetstore.go:108] Matching assets for "protokube$": I1208 11:25:59.027766 34552 assetstore.go:111] /opt/kops/bin/protokube I1208 11:25:59.027795 34552 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I1208 11:25:59.027828 34552 assetstore.go:108] Matching assets for "channels$": I1208 11:25:59.027848 34552 assetstore.go:111] /opt/kops/bin/channels I1208 11:25:59.027861 34552 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I1208 11:25:59.030214 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.030402 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.030442 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.030483 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.030529 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.030568 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.030602 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.030638 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.030663 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.033403 34552 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I1208 11:25:59.036912 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.037124 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.037164 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.037203 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.037251 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.037285 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.037317 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.037356 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.037381 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.040052 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.042674 34552 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I1208 11:25:59.043239 34552 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:59.043423 34552 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:59.043458 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:59.043496 34552 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:59.043542 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:59.043576 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:59.043609 34552 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:59.043643 34552 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:59.043668 34552 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:59.046370 34552 zones.go:93] ListHostedZonesPages { } I1208 11:25:59.048995 34552 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 2.306s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.061s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.080s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.049s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.069s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E1208 11:25:41.141408 28694 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E1208 11:25:41.141591 28694 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.033s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I1208 11:25:49.824416 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824522 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824562 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824594 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824625 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I1208 11:25:49.824834 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824873 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824908 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824938 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.824971 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825003 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825036 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825063 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825093 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825120 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825146 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825176 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825216 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825244 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825269 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825309 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825338 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825363 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1208 11:25:49.825388 32611 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I1208 11:25:49.825533 32611 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I1208 11:25:49.825591 32611 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.144s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I1208 11:25:42.595767 30351 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1208 11:25:42.596877 30351 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1208 11:25:42.598037 30351 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1208 11:25:42.598751 30351 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1208 11:25:42.599901 30351 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1208 11:25:42.600249 30351 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1208 11:25:42.600854 30351 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1208 11:25:42.600964 30351 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1208 11:25:42.601736 30351 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I1208 11:25:42.601813 30351 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I1208 11:25:42.602201 30351 health.go:33] status conditions not found for /v1, Kind=Namespace I1208 11:25:42.602484 30351 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I1208 11:25:42.602530 30351 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I1208 11:25:42.602789 30351 health.go:33] status conditions not found for /v1, Kind=ConfigMap I1208 11:25:42.602999 30351 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I1208 11:25:42.603064 30351 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I1208 11:25:42.603883 30351 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I1208 11:25:42.603947 30351 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.086s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W1208 11:25:42.216320 28695 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (1.04s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W1208 11:25:42.217252 28695 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W1208 11:25:42.218243 28695 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W1208 11:25:42.219137 28695 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W1208 11:25:42.220184 28695 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W1208 11:25:42.221017 28695 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W1208 11:25:42.222057 28695 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E1208 11:25:42.222813 28695 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E1208 11:25:42.222828 28695 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E1208 11:25:42.222837 28695 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E1208 11:25:42.222847 28695 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 1.107s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.060s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.139s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.044s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.162s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.043s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.039s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.019s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.070s === RUN TestFlagToFalse I1208 11:25:42.876257 30582 featureflag_test.go:32] Created flag Unittest1 I1208 11:25:42.876311 30582 featureflag.go:160] FeatureFlag "UnitTest1"=false I1208 11:25:42.876323 30582 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I1208 11:25:42.876391 30582 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.053s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.068s === RUN TestRenderInstanceGroupZones W1208 11:25:45.266735 31215 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.055s === RUN TestWarmPoolOnlyRoll I1208 11:26:07.630081 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.630161 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.630187 34987 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I1208 11:26:07.630634 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.630661 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.631756 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.631774 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.637089 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.637137 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.637154 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.637165 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.638824 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.639431 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.639461 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.640667 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.640693 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.645938 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.645998 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:07.646973 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.647011 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.648244 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.648274 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.653512 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.653563 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.653582 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.653595 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.654598 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.655382 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.655436 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.656643 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.656678 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.661956 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.662006 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.662925 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.662964 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.664175 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.664211 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.669494 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.669545 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.670467 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.670504 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.671726 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.671761 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.676997 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.677028 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.677047 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.677062 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:07.678044 34987 instancegroups.go:407] Draining the node: "node-2a.local". I1208 11:26:07.678903 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.678940 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.680148 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.680191 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.685480 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.685538 34987 instancegroups.go:407] Draining the node: "node-2b.local". I1208 11:26:07.686417 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:07.686458 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.687671 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.687706 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.692983 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.693034 34987 instancegroups.go:407] Draining the node: "node-2c.local". I1208 11:26:07.693868 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:07.693905 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.695109 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.695144 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.700412 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.700462 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I1208 11:26:07.701065 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.701103 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.702290 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.702348 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.702397 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.702412 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.702439 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.703646 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.703697 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.703713 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.703741 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.704947 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.704981 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.705027 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.705042 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.705067 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.706283 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.706334 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.706350 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.706378 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.707585 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.707636 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.707651 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.707678 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.708865 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.708893 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.708940 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.708956 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.708983 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.710181 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.710232 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.710247 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:07.710275 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.711472 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1208 11:26:07.711524 34987 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1208 11:26:07.711540 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:07.711566 34987 instancegroups.go:443] waiting for 1ms after terminating instance W1208 11:26:07.712757 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1208 11:26:07.712797 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I1208 11:26:07.713105 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.713137 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.714342 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.714385 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.714414 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.714465 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.714490 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.714507 34987 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.714521 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.715282 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.716165 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.716204 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.717426 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.717467 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.717489 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.717529 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:07.718368 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.718405 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.719610 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.719652 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.719673 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.719694 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.719714 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.719728 34987 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.719742 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.720803 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.721588 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.721623 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.722811 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.722854 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.722876 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.722915 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.723767 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.723803 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.725010 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.725051 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.725073 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.725112 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.725972 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.726010 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.727233 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.727275 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.727296 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.727311 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.727331 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.727345 34987 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.727359 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:07.728360 34987 instancegroups.go:407] Draining the node: "node-2a.local". I1208 11:26:07.729252 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.729290 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.730507 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.730547 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.730569 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.730608 34987 instancegroups.go:407] Draining the node: "node-2b.local". I1208 11:26:07.731445 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:07.732406 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.733630 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.733671 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.733693 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.733733 34987 instancegroups.go:407] Draining the node: "node-2c.local". I1208 11:26:07.734547 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:07.734587 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.735779 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.735820 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1208 11:26:07.735842 34987 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1208 11:26:07.735860 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I1208 11:26:07.736108 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I1208 11:26:07.736402 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.736434 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.737637 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.737672 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.742960 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.743022 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.743052 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.743065 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.743806 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.744553 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.744589 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.745781 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.745817 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.751088 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.751139 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:07.752019 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.752057 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.753288 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.753322 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.758583 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.758622 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.758643 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.758657 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.759694 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.760457 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.760494 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.761699 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.761734 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.767012 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.767063 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.767863 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.767899 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.769088 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.769122 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.774400 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.774456 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.775286 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.775323 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.776549 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.776584 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.781838 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.781870 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.781899 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.781915 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:07.782948 34987 instancegroups.go:407] Draining the node: "node-2a.local". I1208 11:26:07.783745 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.783781 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.784997 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.785031 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.790313 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.790379 34987 instancegroups.go:407] Draining the node: "node-2b.local". I1208 11:26:07.791226 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:07.791264 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.792456 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.792490 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.797764 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.797818 34987 instancegroups.go:407] Draining the node: "node-2c.local". I1208 11:26:07.798641 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:07.798677 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.799896 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.799930 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.805201 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.805235 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I1208 11:26:07.805355 34987 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I1208 11:26:07.805803 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.805838 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.807046 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.807086 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.807106 34987 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I1208 11:26:07.807438 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.807470 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.808677 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.808713 34987 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1208 11:26:07.808751 34987 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I1208 11:26:07.809006 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.809037 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.809053 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.809080 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1208 11:26:07.809095 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I1208 11:26:07.809239 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.809264 34987 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1208 11:26:07.809279 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.809296 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1208 11:26:07.809310 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I1208 11:26:07.809512 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.809542 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.810738 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.810779 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.816059 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.816119 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.816144 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.816158 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.816883 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.817618 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.817653 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.818819 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.818865 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.818886 34987 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I1208 11:26:07.819161 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.819191 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.820407 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.820447 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.825718 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.825779 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.825803 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.825816 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.826585 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.827322 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.827359 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.828565 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.828607 34987 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1208 11:26:07.828629 34987 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I1208 11:26:07.828914 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.828941 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.828955 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.829973 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.830727 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.830761 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.831975 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.832011 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.837291 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.837346 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.838178 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.838215 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.839426 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.839462 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.844726 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.844776 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.845604 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.845641 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.846864 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.846900 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.852539 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.852580 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I1208 11:26:07.854152 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.854227 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.854271 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.854315 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1208 11:26:07.854341 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I1208 11:26:07.855228 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.855271 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.855294 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.855316 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1208 11:26:07.855332 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I1208 11:26:07.855579 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.855599 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.855610 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.855621 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1208 11:26:07.855632 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I1208 11:26:07.855738 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.855757 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.855768 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.856887 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.857701 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.857741 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.858957 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.859004 34987 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1208 11:26:07.859026 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.859048 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I1208 11:26:07.859068 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I1208 11:26:07.859292 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.859323 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.859336 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.860417 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.861258 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.861293 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.862490 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.862536 34987 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1208 11:26:07.862558 34987 instancegroups.go:482] Cluster did not validate within 0s E1208 11:26:07.862581 34987 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I1208 11:26:07.862596 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I1208 11:26:07.862903 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.862944 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.864175 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.864226 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.865432 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.870733 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.871909 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.877197 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.878359 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.883602 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.883666 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.883699 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.883714 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.884456 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.885279 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.885315 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.886570 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.886616 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.891889 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.891944 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:07.892791 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.892828 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.894065 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.894119 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.899473 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.899515 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.899549 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.899564 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.900597 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.901428 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.901464 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.902672 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.902722 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.908034 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.908088 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.909015 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.909053 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.910291 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.910341 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.915628 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.915688 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.916518 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.916555 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.917742 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.917792 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.923103 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.923134 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.923163 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.923179 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:07.924201 34987 instancegroups.go:407] Draining the node: "node-2a.local". I1208 11:26:07.924976 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.925011 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.926248 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.926298 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.931604 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.931657 34987 instancegroups.go:407] Draining the node: "node-2b.local". I1208 11:26:07.932567 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:07.932605 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.933822 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.933872 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.939149 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.939202 34987 instancegroups.go:407] Draining the node: "node-2c.local". I1208 11:26:07.940086 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:07.940123 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.941349 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.941398 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.946678 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.946713 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I1208 11:26:07.947001 34987 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1208 11:26:07.947036 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.948269 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.948313 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.949506 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.950679 34987 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1208 11:26:07.951843 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.957084 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.957152 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.957186 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.957201 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:07.957983 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:07.958784 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:07.958823 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.960048 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.960085 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.965378 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.965447 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:07.966278 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:07.966315 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.967534 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.967571 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.972817 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.972855 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.972878 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.972893 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:07.973874 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:07.974639 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:07.974676 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.975889 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.975927 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.981203 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.981257 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:07.982167 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:07.982204 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.983437 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.983473 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.988721 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.988791 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:07.989632 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:07.989667 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.990900 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.990936 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:07.996214 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.996246 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.996269 34987 instancegroups.go:510] Cluster validated. I1208 11:26:07.996283 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:07.997292 34987 instancegroups.go:407] Draining the node: "node-2a.local". I1208 11:26:07.998067 34987 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1208 11:26:07.998103 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:07.999316 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:07.999354 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.004633 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.004687 34987 instancegroups.go:407] Draining the node: "node-2b.local". I1208 11:26:08.005570 34987 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1208 11:26:08.005608 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.006814 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.006850 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.012147 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.012200 34987 instancegroups.go:407] Draining the node: "node-2c.local". I1208 11:26:08.013050 34987 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1208 11:26:08.013086 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.014321 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.014358 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.019631 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.019666 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I1208 11:26:08.020348 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.020375 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.020387 34987 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1208 11:26:08.021175 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:08.021953 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:08.021988 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.023244 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.023280 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.028555 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.028607 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:08.029456 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:08.029493 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.030706 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.030742 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.036012 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.036045 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I1208 11:26:08.036315 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.036344 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.036357 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:08.037076 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:08.037828 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:08.037863 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.039069 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.039107 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.044378 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.044429 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:08.045245 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:08.045281 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.046510 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.046554 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.051825 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.051857 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I1208 11:26:08.052204 34987 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W1208 11:26:08.052247 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1208 11:26:08.052270 34987 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W1208 11:26:08.052285 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1208 11:26:08.052308 34987 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W1208 11:26:08.052318 34987 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1208 11:26:08.052338 34987 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I1208 11:26:08.052354 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I1208 11:26:08.052602 34987 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I1208 11:26:08.052658 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.053874 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.053911 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.059171 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.059209 34987 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I1208 11:26:08.059255 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.059277 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.059317 34987 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1208 11:26:08.060081 34987 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I1208 11:26:08.060114 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.060134 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.060147 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:08.061047 34987 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1208 11:26:08.061107 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.062298 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.062336 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.067601 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.067641 34987 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I1208 11:26:08.067657 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.067675 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.067689 34987 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1208 11:26:08.068683 34987 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I1208 11:26:08.068764 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.069982 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.070017 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:08.075285 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.075323 34987 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I1208 11:26:08.075342 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I1208 11:26:08.075595 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.075629 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.075642 34987 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I1208 11:26:08.077754 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:08.078523 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:08.078571 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.079801 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.079850 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.079901 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:08.079966 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:08.081369 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:08.081695 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:08.081747 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.082981 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.083045 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.183584 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.183640 34987 instancegroups.go:407] Draining the node: "node-1e.local". I1208 11:26:08.183664 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:08.184958 34987 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1208 11:26:08.185289 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:08.185340 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.186571 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.186615 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.186657 34987 instancegroups.go:407] Draining the node: "node-1f.local". I1208 11:26:08.187480 34987 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1208 11:26:08.187526 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.188745 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.188816 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.289393 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.289476 34987 instancegroups.go:407] Draining the node: "node-1g.local". I1208 11:26:08.290470 34987 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1208 11:26:08.290524 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.291753 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.291799 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.291818 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I1208 11:26:08.292089 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.292125 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.292139 34987 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1208 11:26:08.294029 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:08.294096 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:08.295306 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:08.295617 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:08.295661 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.296859 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.296931 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.397549 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.397618 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:08.397705 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:08.398979 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:08.399262 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:08.399305 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.400515 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.400558 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.400598 34987 instancegroups.go:407] Draining the node: "node-1e.local". I1208 11:26:08.401419 34987 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1208 11:26:08.401463 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.402660 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.402721 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.503198 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.503274 34987 instancegroups.go:407] Draining the node: "node-1f.local". I1208 11:26:08.504231 34987 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1208 11:26:08.504281 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.505479 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.505526 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.505546 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I1208 11:26:08.505820 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.505856 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.505871 34987 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I1208 11:26:08.509997 34987 instancegroups.go:407] Draining the node: "master-1a.local". I1208 11:26:08.510874 34987 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1208 11:26:08.510936 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.512148 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.512194 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.512251 34987 instancegroups.go:407] Draining the node: "master-1c.local". I1208 11:26:08.512312 34987 instancegroups.go:407] Draining the node: "master-1b.local". I1208 11:26:08.513377 34987 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I1208 11:26:08.513665 34987 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1208 11:26:08.513715 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.514935 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.515057 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.615582 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.615636 34987 instancegroups.go:407] Draining the node: "master-1e.local". I1208 11:26:08.615680 34987 instancegroups.go:407] Draining the node: "master-1d.local". I1208 11:26:08.617007 34987 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I1208 11:26:08.617340 34987 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I1208 11:26:08.617389 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.618620 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.618663 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.618703 34987 instancegroups.go:407] Draining the node: "master-1f.local". I1208 11:26:08.619620 34987 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I1208 11:26:08.619665 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.620878 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.620939 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.721438 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.721488 34987 instancegroups.go:407] Draining the node: "master-1g.local". I1208 11:26:08.722803 34987 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I1208 11:26:08.722849 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.724061 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.724104 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.724123 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I1208 11:26:08.724412 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.724448 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.724464 34987 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1208 11:26:08.726318 34987 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1208 11:26:08.726388 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.727602 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.727646 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.727660 34987 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1208 11:26:08.727715 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.728895 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.728939 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.728983 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:08.729064 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:08.730331 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:08.730680 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:08.730728 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.731991 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.732056 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.832674 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.832727 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:08.832753 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:08.834143 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:08.834431 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:08.834474 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.835670 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.835717 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.835764 34987 instancegroups.go:407] Draining the node: "node-1e.local". I1208 11:26:08.836647 34987 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1208 11:26:08.836704 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.837935 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.838005 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.938599 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.938648 34987 instancegroups.go:407] Draining the node: "node-1f.local". I1208 11:26:08.939516 34987 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1208 11:26:08.939566 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.940766 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.940811 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.940830 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I1208 11:26:08.941140 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.941180 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.941194 34987 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1208 11:26:08.943212 34987 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1208 11:26:08.943267 34987 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1208 11:26:08.943327 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:08.944563 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.944609 34987 instancegroups.go:510] Cluster validated. I1208 11:26:08.944656 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:08.944724 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:08.946006 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:08.946306 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:08.946351 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:08.947547 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:08.947614 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.048175 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.048236 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:09.048257 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:09.049595 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:09.049869 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:09.049911 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.051115 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.051160 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.051206 34987 instancegroups.go:407] Draining the node: "node-1e.local". I1208 11:26:09.052122 34987 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1208 11:26:09.052176 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.053409 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.053471 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.154369 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.154419 34987 instancegroups.go:407] Draining the node: "node-1f.local". I1208 11:26:09.155285 34987 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1208 11:26:09.155333 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.156587 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.156634 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.156653 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I1208 11:26:09.156933 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.156969 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.156983 34987 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1208 11:26:09.157738 34987 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1208 11:26:09.157779 34987 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I1208 11:26:09.157817 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:09.159033 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.159070 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.164356 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.164416 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:09.164442 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:09.165823 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:09.165865 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.166134 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:09.166169 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.167106 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.167141 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.172497 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.172534 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I1208 11:26:09.172757 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.172786 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.172799 34987 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1208 11:26:09.173518 34987 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E1208 11:26:09.173584 34987 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I1208 11:26:09.173599 34987 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E1208 11:26:09.173638 34987 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I1208 11:26:09.173672 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:09.174876 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.174910 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.180138 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.180192 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:09.180227 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:09.181541 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:09.181581 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.181841 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:09.181873 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.182804 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.182842 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.188503 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.188539 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I1208 11:26:09.188780 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.188816 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.188829 34987 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I1208 11:26:09.190086 34987 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1208 11:26:09.190146 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:09.191367 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.191415 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.191431 34987 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1208 11:26:09.191502 34987 instancegroups.go:179] waiting for 1ms after detaching instance I1208 11:26:09.192741 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.192786 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.192839 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:09.192912 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:09.192980 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:09.194600 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:09.194652 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.194921 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:09.194961 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.195235 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:09.195276 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.195814 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.296402 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.296457 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:09.297393 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:09.297445 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.298691 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.298774 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.298798 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I1208 11:26:09.299024 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.299056 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.299070 34987 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I1208 11:26:09.301191 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:09.301948 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:09.301991 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.303190 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.303238 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.303295 34987 instancegroups.go:407] Draining the node: "node-1e.local". I1208 11:26:09.303348 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:09.304680 34987 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1208 11:26:09.305045 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:09.305098 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.306306 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.306370 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.407398 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.407459 34987 instancegroups.go:407] Draining the node: "node-1g.local". I1208 11:26:09.407480 34987 instancegroups.go:407] Draining the node: "node-1f.local". I1208 11:26:09.408970 34987 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1208 11:26:09.409264 34987 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1208 11:26:09.409307 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.410526 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.410573 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.410616 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:09.411479 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:09.411526 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.412764 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.412827 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.513428 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.513478 34987 instancegroups.go:407] Draining the node: "node-1d.local". I1208 11:26:09.514330 34987 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1208 11:26:09.514373 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.515665 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.515712 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.515738 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I1208 11:26:09.515959 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.515987 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.516002 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1208 11:26:09.516021 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1208 11:26:09.516037 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1208 11:26:09.516056 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I1208 11:26:09.516213 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.516240 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.516254 34987 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1208 11:26:09.517335 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1208 11:26:09.517373 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1208 11:26:09.517391 34987 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1208 11:26:09.517434 34987 instancegroups.go:407] Draining the node: "node-1a.local". I1208 11:26:09.518217 34987 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1208 11:26:09.518255 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.519484 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.519520 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.524779 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.524832 34987 instancegroups.go:407] Draining the node: "node-1b.local". I1208 11:26:09.525748 34987 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1208 11:26:09.525790 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.527022 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.527056 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.532309 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.532363 34987 instancegroups.go:407] Draining the node: "node-1c.local". I1208 11:26:09.533198 34987 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1208 11:26:09.533238 34987 instancegroups.go:443] waiting for 1ms after terminating instance I1208 11:26:09.534480 34987 instancegroups.go:477] Validating the cluster. I1208 11:26:09.534514 34987 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1208 11:26:09.539777 34987 instancegroups.go:510] Cluster validated. I1208 11:26:09.539812 34987 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.991s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.027s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.042s === RUN TestParse E1208 11:25:44.184577 30917 versions.go:96] unable to parse Kubernetes version "" E1208 11:25:44.184654 30917 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.019s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.100s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.28s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.08s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.02s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.07s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.453s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.087s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.061s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.154s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I1208 11:25:51.357455 32881 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.108s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I1208 11:25:54.517512 33666 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W1208 11:25:54.624252 33666 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W1208 11:25:54.639537 33666 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.15s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.04s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.380s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I1208 11:25:53.993226 33529 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.135s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.110s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.055s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.153s === RUN TestServerGroupBuilder I1208 11:25:56.110806 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.126253 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.144936 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.157132 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.238055 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.257645 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.277793 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.294023 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.301651 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.377612 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.384232 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.391855 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.398558 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.410495 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:56.416850 34384 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.31s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.385s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.054s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.055s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.052s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.45s) --- PASS: TestIssueCert/ca (0.47s) --- PASS: TestIssueCert/client (0.77s) --- PASS: TestIssueCert/clientOneYear (0.20s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.528s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.080s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I1208 11:25:55.530849 34197 routetable.go:53] DescribeRouteTables: { } W1208 11:25:55.530967 34197 tags.go:67] cluster tag not found on route-table:rtb-1234 I1208 11:25:55.530985 34197 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I1208 11:25:55.531128 34197 iaminstanceprofile.go:193] ListInstanceProfiles: { } W1208 11:25:55.531204 34197 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I1208 11:25:55.531311 34197 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I1208 11:25:55.531419 34197 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I1208 11:25:55.531515 34197 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I1208 11:25:55.531614 34197 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I1208 11:25:55.531729 34197 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I1208 11:25:55.531782 34197 volumes.go:128] DescribeVolumes: { } W1208 11:25:55.531820 34197 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I1208 11:25:55.532012 34197 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:25:55.532247 34197 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:55.532291 34197 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:25:55.532368 34197 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:25:55.532429 34197 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:25:55.532479 34197 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:25:55.532507 34197 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:25:55.532543 34197 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:25:55.532577 34197 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:25:55.532608 34197 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I1208 11:25:55.532645 34197 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I1208 11:25:55.532681 34197 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I1208 11:25:55.532730 34197 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I1208 11:25:55.532765 34197 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I1208 11:25:55.532810 34197 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I1208 11:25:55.532854 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I1208 11:25:55.532897 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I1208 11:25:55.532930 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I1208 11:25:55.532962 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I1208 11:25:55.532998 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I1208 11:25:55.533029 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I1208 11:25:55.533063 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I1208 11:25:55.533091 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I1208 11:25:55.533126 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I1208 11:25:55.533155 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I1208 11:25:55.533192 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I1208 11:25:55.533221 34197 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.137s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.077s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.163s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.017s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.028s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.010s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.039s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.007s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.02s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.01s) PASS ok k8s.io/kops/pkg/util/templater 0.125s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.066s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.031s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W1208 11:25:48.289841 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.289898 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.289910 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.289920 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.289929 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.289939 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.289948 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.289957 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.289966 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.289976 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.289986 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.290001 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.290010 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.290020 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.290029 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.290038 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.292436 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.292511 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.292606 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.292645 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.292728 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.292764 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.292847 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.292865 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.292924 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.292940 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293016 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293056 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293154 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293191 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293279 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293316 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293397 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293435 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293533 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293569 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293665 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293710 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293792 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293828 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.293909 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.293945 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294027 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294063 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294145 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294180 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294264 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294301 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294367 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294385 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294444 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294482 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294545 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294561 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294661 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294677 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294736 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294752 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294809 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294824 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294878 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294893 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.294949 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.294963 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300228 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300300 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300410 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300426 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300491 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300507 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300565 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300580 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300636 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300651 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300719 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300735 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300792 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300807 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300871 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300887 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.300954 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.300970 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301028 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301043 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301125 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301141 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301210 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301226 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301307 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301323 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301402 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301417 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301474 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301489 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301551 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301565 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301665 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301682 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301742 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301757 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301814 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301828 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301884 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301900 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.301954 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.301969 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302026 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302041 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302112 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302128 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302190 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302212 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302273 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302288 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302345 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302360 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302425 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302441 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302499 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302514 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302574 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302588 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302643 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302657 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302714 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302729 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302787 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302804 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302863 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302879 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.302936 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.302951 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303005 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303019 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303073 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303088 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303142 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303157 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303215 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303230 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303306 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303321 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303389 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303404 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303459 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303474 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303531 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303546 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303600 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303615 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303685 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303700 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303758 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303773 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303827 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303842 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303896 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303911 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.303969 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.303984 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304049 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304065 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304129 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304146 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304206 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304221 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304274 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304288 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304345 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304359 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304414 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304428 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304484 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304499 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304559 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304574 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304631 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304645 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304701 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304716 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304770 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304784 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304864 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304879 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.304938 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.304952 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305008 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305022 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305076 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305091 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305148 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305163 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305219 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305234 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305871 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305891 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.305947 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.305962 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306018 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306033 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306088 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306103 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306161 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306176 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306235 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306252 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306315 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306329 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306383 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306397 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306450 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306464 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.306516 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.306530 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.02s) === RUN TestRecoversFromBadNesting W1208 11:25:48.307200 32360 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1208 11:25:48.307214 32360 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1208 11:25:48.307224 32360 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1208 11:25:48.307232 32360 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1208 11:25:48.307241 32360 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1208 11:25:48.307248 32360 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1208 11:25:48.307255 32360 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1208 11:25:48.307281 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307294 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307304 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.307312 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307321 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.307329 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307341 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.307351 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307360 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.307368 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307378 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307387 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307395 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307404 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.307413 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.307422 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316127 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316160 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316228 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316244 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316304 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316317 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316376 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316389 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316472 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316485 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316552 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316567 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316626 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316639 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316693 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316706 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316766 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316780 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316847 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316861 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316924 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.316937 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.316997 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317010 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317100 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317114 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317176 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317189 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317250 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317263 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317320 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317335 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317405 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317419 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317477 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317490 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317547 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317561 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317626 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317639 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317695 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317708 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317771 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317784 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317847 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317862 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317924 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.317938 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.317994 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318007 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318069 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318082 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318139 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318152 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318226 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318240 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318317 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318330 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318390 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318403 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318459 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318472 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318528 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318544 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318612 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318628 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318698 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318713 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318789 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318804 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318868 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318888 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.318953 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.318973 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319034 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319051 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319115 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319132 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319195 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319212 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319273 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319290 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319351 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319367 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319446 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319464 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319528 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319545 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319607 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319624 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319685 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319702 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319769 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319785 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319852 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319868 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.319937 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.319954 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320013 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320031 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320091 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320108 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320179 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320197 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320279 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320297 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320359 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320376 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320436 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320452 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320511 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320528 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320586 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320603 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320666 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320683 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320744 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320761 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320826 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320846 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320917 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.320936 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.320997 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321015 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321079 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321096 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321157 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321174 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321232 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321327 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321411 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321447 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321525 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321559 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321639 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321674 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321755 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321791 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321867 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.321901 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.321986 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322022 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322108 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322145 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322223 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322271 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322369 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322412 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322496 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322532 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322615 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322666 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322771 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322808 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.322887 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.322923 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323002 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323040 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323119 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323154 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323237 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323273 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323351 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323386 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323465 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323499 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323581 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323617 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323698 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323734 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323811 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323846 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.323924 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.323960 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324054 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324098 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324184 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324223 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324305 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324341 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324419 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324453 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324539 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324574 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324681 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324718 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324796 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324832 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.324913 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.324947 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.325024 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.325059 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.325136 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.325171 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.325249 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.325286 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1208 11:25:48.325378 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1208 11:25:48.325430 32360 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.048s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.064s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.098s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.01s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.115s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.12s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.03s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.08s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.02s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.01s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.02s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.318s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I1208 11:25:55.138378 34100 files.go:57] Writing file "/tmp/TestWriteFile2605614233/001/SubDir/test1.tmp" I1208 11:25:55.138570 34100 files.go:113] Changing file mode for "/tmp/TestWriteFile2605614233/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.052s === RUN TestBootstrapChannelBuilder_BuildTasks I1208 11:26:01.273074 34915 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:01.273345 34915 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:01.273395 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:01.273433 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:01.273490 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:01.273540 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:01.273557 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:01.273590 34915 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:01.273619 34915 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:01.280840 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.328136 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.447160 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.507972 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.595378 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.695848 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.757924 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.837023 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:01.964455 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:02.501108 34915 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.29s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I1208 11:26:02.558782 34915 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:02.558948 34915 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:02.559016 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:02.559051 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:02.559086 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:02.559114 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:02.559132 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:02.559158 34915 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:02.559177 34915 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:02.559194 34915 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I1208 11:26:02.560359 34915 zones.go:93] ListHostedZonesPages { } I1208 11:26:02.639456 34915 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.08s) === RUN TestBootstrapChannelBuilder_AWSCloudController I1208 11:26:02.639770 34915 featureflag.go:160] FeatureFlag "ImageDigest"=false I1208 11:26:02.640034 34915 internetgateways.go:66] CreateInternetGateway: { } I1208 11:26:02.640115 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1208 11:26:02.640163 34915 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1208 11:26:02.640213 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1208 11:26:02.640251 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1208 11:26:02.640282 34915 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1208 11:26:02.640320 34915 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1208 11:26:02.640358 34915 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1208 11:26:02.642244 34915 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.17s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.11s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W1208 11:26:03.084404 34915 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.01s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I1208 11:26:03.138474 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138507 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138523 34915 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1208 11:26:03.138549 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1208 11:26:03.138564 34915 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I1208 11:26:03.138589 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1208 11:26:03.138617 34915 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I1208 11:26:03.138643 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138661 34915 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1208 11:26:03.138684 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138701 34915 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1208 11:26:03.138727 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138776 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138807 34915 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1208 11:26:03.138823 34915 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1208 11:26:03.138837 34915 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I1208 11:26:03.138851 34915 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I1208 11:26:03.138865 34915 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I1208 11:26:03.138881 34915 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1208 11:26:03.138896 34915 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I1208 11:26:03.138911 34915 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I1208 11:26:03.140359 34915 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.956s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I1208 11:25:58.197900 34710 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1369437763/001 I1208 11:25:58.200029 34710 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1369437763/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I1208 11:25:58.207379 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1208 11:25:58.207481 34710 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1208 11:25:58.207572 34710 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I1208 11:25:58.207875 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.208004 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.208077 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.208100 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.208280 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.208321 34710 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1208 11:25:58.208363 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.208438 34710 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1208 11:25:58.208533 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1208 11:25:58.208683 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.208756 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.208808 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.208830 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.208884 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.208938 34710 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1208 11:25:58.209006 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I1208 11:25:58.209367 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.209426 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.209555 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.209602 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.209630 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.209660 34710 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1208 11:25:58.209808 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.210015 34710 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1208 11:25:58.210067 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.210097 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.210130 34710 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1208 11:25:58.210205 34710 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.210387 34710 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I1208 11:25:58.210458 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.210489 34710 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I1208 11:25:58.210534 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1208 11:25:58.210700 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.210751 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.210804 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.210824 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.210856 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.210884 34710 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1208 11:25:58.210926 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.210996 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.211026 34710 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1208 11:25:58.211079 34710 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.211137 34710 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I1208 11:25:58.211178 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1208 11:25:58.211233 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I1208 11:25:58.211403 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1208 11:25:58.211450 34710 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1208 11:25:58.211479 34710 internetgateways.go:66] CreateInternetGateway: { } I1208 11:25:58.211505 34710 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I1208 11:25:58.211645 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.211691 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.211744 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.211765 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.211843 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.211876 34710 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1208 11:25:58.211914 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.211997 34710 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1208 11:25:58.212102 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1208 11:25:58.212258 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.212307 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.212353 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.212373 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.212413 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.212456 34710 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1208 11:25:58.212539 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I1208 11:25:58.214859 34710 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1237416760/001 I1208 11:25:58.216676 34710 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1237416760/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I1208 11:25:58.217343 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.217406 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.217526 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.217578 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.217608 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.217637 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.217711 34710 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1208 11:25:58.217928 34710 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I1208 11:25:58.217992 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1208 11:25:58.218030 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1208 11:25:58.218200 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.218252 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.218306 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.218327 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.218359 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.218388 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.218437 34710 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1208 11:25:58.218525 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I1208 11:25:58.218986 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.219040 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.219152 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.219198 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.219228 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.219257 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.219305 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.219426 34710 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1208 11:25:58.219478 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.219506 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.219535 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1208 11:25:58.219685 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.219736 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.219788 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.219808 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.219839 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.219867 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.219912 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.219974 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.220004 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I1208 11:25:58.220312 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.220363 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.220464 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.220511 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.220538 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.220567 34710 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1208 11:25:58.220625 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.220776 34710 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1208 11:25:58.220760 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.220937 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.220974 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.220998 34710 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:25:58.221034 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1208 11:25:58.221199 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.221250 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.221302 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.221322 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.221353 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.221381 34710 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1208 11:25:58.221421 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.221467 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.221525 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.221547 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I1208 11:25:58.222127 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.222176 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.222237 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.222265 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.222281 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.222304 34710 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1208 11:25:58.222335 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.222441 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.222478 34710 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1208 11:25:58.222526 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.222550 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.222581 34710 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1208 11:25:58.222610 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1208 11:25:58.222753 34710 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1208 11:25:58.222787 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.222819 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.222831 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.222854 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.222874 34710 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1208 11:25:58.222930 34710 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1208 11:25:58.222993 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.223057 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.223082 34710 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I1208 11:25:58.223251 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I1208 11:25:58.223333 34710 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I1208 11:25:58.223488 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.223538 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.223583 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.223603 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.223682 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.223717 34710 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1208 11:25:58.223755 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.223798 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.223913 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1208 11:25:58.223943 34710 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1208 11:25:58.223979 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1208 11:25:58.224149 34710 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1208 11:25:58.224208 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.224267 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.224287 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.224326 34710 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1208 11:25:58.224371 34710 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1208 11:25:58.224436 34710 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I1208 11:25:58.224696 34710 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.224749 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.224842 34710 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1208 11:25:58.224887 34710 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1208 11:25:58.224918 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.224945 34710 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.225068 34710 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.225116 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1208 11:25:58.225167 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.225187 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.225220 34710 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1208 11:25:58.225249 34710 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I1208 11:25:58.225511 34710 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I1208 11:25:58.225661 34710 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.225712 34710 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I1208 11:25:58.225764 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1208 11:25:58.225784 34710 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1208 11:25:58.225823 34710 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.072s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.046s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.048s === RUN TestDiskRenderAzure I1208 11:25:54.307228 33624 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I1208 11:25:54.307513 33624 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I1208 11:25:54.307694 33624 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I1208 11:25:54.308067 33624 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I1208 11:25:54.308231 33624 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I1208 11:25:54.308449 33624 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I1208 11:25:54.308669 33624 resourcegroup.go:103] Creating a new Resource Group with name: rg I1208 11:25:54.308685 33624 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I1208 11:25:54.308986 33624 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I1208 11:25:54.309873 33624 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I1208 11:25:54.310347 33624 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I1208 11:25:54.310730 33624 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I1208 11:25:54.311353 33624 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I1208 11:25:54.311778 33624 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.088s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.084s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I1208 11:25:58.549418 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.549924 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.550052 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.550392 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.550502 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.550630 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I1208 11:25:58.550801 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.550884 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.550982 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.551079 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.551195 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.551237 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I1208 11:25:58.551401 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.551632 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.551733 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.551997 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1208 11:25:58.552134 34754 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1208 11:25:58.552249 34754 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.047s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.044s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.027s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.099s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.011s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.051s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I1208 11:26:00.079356 34822 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I1208 11:26:00.079426 34822 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I1208 11:26:00.079437 34822 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I1208 11:26:00.079448 34822 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I1208 11:26:00.079458 34822 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I1208 11:26:00.079467 34822 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I1208 11:26:00.079477 34822 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W1208 11:26:00.079910 34822 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.061s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.051s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.019s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.003s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.004s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.004s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W1208 11:25:56.638349 34540 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.044s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.031s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.004s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.009s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.020s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W1208 11:26:01.484089 34945 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.067s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 133.6 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r1.apk >>> kops: Build complete at Thu, 08 Dec 2022 11:26:58 +0000 elapsed time 0h 3m 24s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20221208.112336) (2/3) Purging kubernetes (1.25.0-r4) (3/3) Purging go (1.19.4-r0) Executing busybox-1.35.0-r29.trigger OK: 393 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...