>>> kops: Building testing/kops 1.25.0-r1 (using abuild 3.10.0_rc1-r2) started Fri, 07 Oct 2022 22:17:32 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r2) (2/3) Installing go (1.19.2-r0) (3/3) Installing .makedepends-kops (20221007.221733) Executing busybox-1.35.0-r27.trigger OK: 800 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 52 30.7M 52 16.2M 0 0 30.5M 0 0:00:01 --:--:-- 0:00:01 30.5M 100 30.7M 100 30.7M 0 0 44.5M 0 --:--:-- --:--:-- --:--:-- 44.5M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.0.tar.gz >>> kops: Checking sha512sums... kops-1.25.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.0.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.85 go: downloading google.golang.org/api v0.94.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.9.1 go: downloading k8s.io/klog/v2 v2.70.1 go: downloading k8s.io/apimachinery v0.25.0 go: downloading github.com/spf13/cobra v1.5.0 go: downloading golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094 go: downloading k8s.io/api v0.25.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/client-go v0.25.0 go: downloading golang.org/x/crypto v0.0.0-20220824171710-5757bc0c5503 go: downloading github.com/stretchr/testify v1.8.0 go: downloading k8s.io/cli-runtime v0.25.0 go: downloading cloud.google.com/go/compute v1.9.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading helm.sh/helm/v3 v3.9.4 go: downloading k8s.io/kubectl v0.25.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading sigs.k8s.io/controller-runtime v0.12.3 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/prometheus/client_golang v1.13.0 go: downloading k8s.io/component-base v0.25.0 go: downloading github.com/google/go-cmp v0.5.8 go: downloading github.com/digitalocean/godo v1.83.0 go: downloading github.com/hetznercloud/hcloud-go v1.35.2 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.0 go: downloading k8s.io/mount-utils v0.25.0 go: downloading k8s.io/utils v0.0.0-20220823124924-e9cbc92d1a73 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading golang.org/x/sys v0.0.0-20220825204002-c680a09ffe64 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading k8s.io/cloud-provider-aws v1.24.1 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/spotinst/spotinst-sdk-go v1.126.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.11.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/golang/protobuf v1.5.2 go: downloading golang.org/x/text v0.3.7 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading k8s.io/apiextensions-apiserver v0.24.2 go: downloading sigs.k8s.io/gateway-api v0.4.3 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.20 go: downloading k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading google.golang.org/grpc v1.48.0 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading k8s.io/csi-translation-lib v0.25.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading golang.org/x/tools v0.1.8 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading google.golang.org/genproto v0.0.0-20220804142021-4e6b2dfa6612 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.19.0 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/onsi/ginkgo/v2 v2.1.4 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/moby/locker v1.0.1 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.0/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.0/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I1007 22:19:37.784325 19237 addon.go:233] addon test wants to update all nodes I1007 22:19:37.786476 19237 addon.go:233] addon test wants to update worker nodes I1007 22:19:37.786850 19237 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I1007 22:19:37.787324 19237 addon.go:266] installing PKI for "test" I1007 22:19:38.376753 19237 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.28s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.368s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.120s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I1007 22:19:53.087566 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.087842 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.087891 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.087945 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.087999 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.088126 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.088149 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.088198 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.088229 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.088314 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.088365 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.136625 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.136793 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.137179 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.147975 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.148113 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.148155 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.148180 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.148218 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.148238 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.148252 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.148271 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.148283 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.148343 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.148376 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.173003 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.173224 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.173694 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.181037 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.181198 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.181237 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.181276 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.181316 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.181349 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.181371 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.181401 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.181423 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.181504 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.181537 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.218931 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.219136 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.219509 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.226531 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.226679 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.226719 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.226753 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.226788 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.226823 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.226846 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.226876 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.226897 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.226995 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.227029 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.248491 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.248646 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.248922 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.253993 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.254171 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.254223 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.254259 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.254298 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.254341 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.254367 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.254414 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.254439 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.254513 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.254544 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.291818 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.291981 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.292262 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.299542 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.299695 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.299726 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.299752 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.299778 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.299813 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.299827 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.299846 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.299872 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.299919 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.299939 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.325307 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.325447 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.325688 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.330138 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.330296 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.330325 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.330349 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.330376 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.330399 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.330414 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.330434 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.330448 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.330551 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.330588 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.413211 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.413372 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.413642 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.33s) === RUN TestCreateClusterHetzner I1007 22:19:53.422069 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.422297 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.422336 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.422376 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.422417 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.422486 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.422511 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.422540 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.422576 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.422649 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.422693 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1007 22:19:53.470929 23581 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I1007 22:19:53.471005 23581 new_cluster.go:1277] Cloud Provider ID = hetzner I1007 22:19:53.481561 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.481751 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.481791 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.481828 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.481869 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.481903 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.481937 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.481971 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.481994 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.482092 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.482130 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.542288 23581 new_cluster.go:1277] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.12s) === RUN TestCreateClusterOverride I1007 22:19:53.546612 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.546756 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.546785 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.546811 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.546850 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.546918 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.546932 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.546950 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.546963 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.547009 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.547030 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.561526 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.561936 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.562149 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.02s) === RUN TestCreateClusterComplex I1007 22:19:53.568553 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.568689 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.568721 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.568750 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.568778 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.568822 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.568837 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.568856 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.568871 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.568926 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.568951 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.617637 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.617799 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.618068 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.05s) === RUN TestCreateClusterHA I1007 22:19:53.622432 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.622560 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.622589 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.622612 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.622638 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.622682 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.622699 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.622718 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.622733 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.622788 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.622808 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.643530 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.643801 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.643813 23581 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1007 22:19:53.643828 23581 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1007 22:19:53.644141 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.651113 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.651243 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.651276 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.651301 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.651329 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.651354 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.651368 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.651388 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.651404 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.651467 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.651489 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.708856 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.709107 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.709119 23581 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1007 22:19:53.709126 23581 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1007 22:19:53.709379 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.09s) === RUN TestCreateClusterHAGCE I1007 22:19:53.715908 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.716029 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.716055 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.716079 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.716103 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.716146 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.716162 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.716181 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.716195 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.716242 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.716262 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.755365 23581 new_cluster.go:1277] Cloud Provider ID = gce I1007 22:19:53.755470 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.05s) === RUN TestCreateClusterGCE I1007 22:19:53.762117 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.762241 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.762266 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.762289 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.762313 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.762355 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.762369 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.762387 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.762400 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.762448 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.762469 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.807658 23581 new_cluster.go:528] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1007 22:19:53.807735 23581 new_cluster.go:1277] Cloud Provider ID = gce I1007 22:19:53.807820 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.05s) === RUN TestCreateClusterHASharedZone I1007 22:19:53.815061 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.815205 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.815237 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.815268 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.815300 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.815348 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.815363 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.815387 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.815403 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.815462 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.815485 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1007 22:19:53.839096 23581 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I1007 22:19:53.839145 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.839357 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.839637 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.03s) === RUN TestCreateClusterHASharedZones I1007 22:19:53.845748 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.845883 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.845916 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.845942 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.845969 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.846019 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.846034 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.846054 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.846068 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.846130 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.846151 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1007 22:19:53.882564 23581 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I1007 22:19:53.882618 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.882925 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.882945 23581 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1007 22:19:53.883244 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I1007 22:19:53.892865 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.893014 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.893043 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.893069 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.893094 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.893136 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.893151 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.893168 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.893181 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.893228 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.893248 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.925707 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.925919 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.925936 23581 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1007 22:19:53.926241 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.926283 23581 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.04s) === RUN TestCreateClusterPrivateGCE I1007 22:19:53.932591 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.932781 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.932820 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.932847 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.932874 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.932918 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.932933 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.932953 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.932968 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.933017 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.933038 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.961392 23581 new_cluster.go:528] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1007 22:19:53.961449 23581 new_cluster.go:1277] Cloud Provider ID = gce I1007 22:19:53.961518 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.03s) === RUN TestCreateClusterWithNGWSpecified I1007 22:19:53.966864 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.966986 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.967011 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.967038 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.967064 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.967107 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.967122 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.967144 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.967175 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.967223 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.967245 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.987906 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:53.988100 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:53.988114 23581 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1007 22:19:53.988353 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.988379 23581 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.03s) === RUN TestCreateClusterWithINGWSpecified I1007 22:19:53.995091 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.995234 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.995261 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.995296 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.995323 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.995379 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.995393 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.995410 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.995422 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.995479 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:53.995501 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.059113 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.059325 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:54.059344 23581 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1007 22:19:54.059645 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:54.059677 23581 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.07s) === RUN TestCreateClusterSharedVPC I1007 22:19:54.065887 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.066034 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.066065 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.066092 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.066120 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.066174 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.066193 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.066216 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.066234 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.066289 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.066315 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.160037 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.160184 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.160233 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.160277 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.160295 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.160338 23581 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I1007 22:19:54.160646 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.10s) === RUN TestCreateClusterSharedSubnets I1007 22:19:54.168144 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.168319 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.168353 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.168391 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.168426 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.168483 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.168504 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.168528 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.168546 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.168612 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.168641 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.256049 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.256095 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.256163 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.256316 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.256343 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.256371 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.256387 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.256692 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.09s) === RUN TestCreateClusterSharedSubnetsVpcLookup I1007 22:19:54.262084 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.262252 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.262284 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.262308 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.262336 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.262394 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.262411 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.262432 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.262452 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.262508 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.262534 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.321281 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:54.321356 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.321393 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.321501 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.321761 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.321800 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.321848 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.321881 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.322323 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.07s) === RUN TestCreateClusterPrivateSharedSubnets I1007 22:19:54.330789 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.330992 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.331031 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.331068 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.331111 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.331187 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.331214 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.331249 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.331274 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.331355 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.331395 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.376328 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.376372 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.376439 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.376456 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.376482 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.376597 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.376613 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.376633 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:19:54.376644 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1007 22:19:54.376952 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.05s) === RUN TestCreateClusterIPv6 I1007 22:19:54.381624 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.381758 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.381786 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.381808 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.381834 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.381879 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.381894 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.381913 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.381927 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.382006 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.382029 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.424515 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.424666 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:54.425001 23581 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestCreateClusterDifferentAMIs I1007 22:19:54.430000 23581 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1007 22:19:54.430015 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.430148 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.430179 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.430203 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.430230 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.430276 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.430292 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.430312 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.430329 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.430387 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.430408 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.462299 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.462380 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:54.462392 23581 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1007 22:19:54.462694 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:54.470946 23581 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.04s) === RUN TestCreateClusterKarpenter I1007 22:19:54.471030 23581 featureflag.go:160] FeatureFlag "Karpenter"=true I1007 22:19:54.471039 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.471174 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.471206 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.471232 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.471262 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.471309 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.471322 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.471341 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.471356 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.471413 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.471434 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1007 22:19:54.513267 23581 new_cluster.go:1277] Cloud Provider ID = aws I1007 22:19:54.513429 23581 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1007 22:19:54.513442 23581 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1007 22:19:54.513450 23581 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1007 22:19:54.513779 23581 zones.go:93] ListHostedZonesPages { } I1007 22:19:54.519470 23581 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.05s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I1007 22:19:54.519743 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.519875 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.519907 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.519963 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.519993 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.520017 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.520033 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.520053 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.520067 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.524827 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I1007 22:19:54.525928 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.526088 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.526120 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.526144 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.526171 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.526212 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.526225 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.526248 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.526262 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.540848 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:19:54.542728 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:19:54.542791 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:19:57.404740 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:00.133540 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:00.133710 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:00.135191 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:00.135633 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:00.135636 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:00.135657 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:00.135719 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:00.136508 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:00.141285 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:00.141922 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.142077 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.142614 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.145059 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.145181 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:00.145596 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:00.145817 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:00.194117 23581 target.go:112] Terraform output is in /tmp/test3533826028/out I1007 22:20:00.197897 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:00.198155 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:00.198215 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:00.198287 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:00.198359 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:00.198408 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:00.198440 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:00.198497 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:00.198526 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:00.222345 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:00.225768 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.225884 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:00.270607 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:00.365132 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:00.365534 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:00.365696 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:00.365737 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:00.365847 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:00.365870 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:00.366049 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:00.366064 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:00.366155 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W1007 22:20:00.366248 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:00.371402 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:00.376808 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:00.377116 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.377270 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.377649 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.378263 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.378332 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:00.378462 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:00.378535 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:00.380642 23581 target.go:168] Cloudformation output is in /tmp/test731341685/out --- PASS: TestMinimal (5.86s) === RUN TestMinimal_v1_23 I1007 22:20:00.385091 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:00.385191 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:00.385215 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:00.385257 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:00.385279 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:00.385301 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:00.385314 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:00.385332 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:00.385345 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:00.399965 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1007 22:20:00.401550 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:00.401598 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.703522 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:01.845404 23581 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1007 22:20:01.845935 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:01.846903 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:01.846944 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:01.847050 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:01.847190 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:01.847990 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:01.851956 23581 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1007 22:20:01.860295 23581 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1007 22:20:01.861073 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.861255 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.861861 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.862831 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.862898 23581 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1007 22:20:01.863087 23581 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1007 22:20:01.863190 23581 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1007 22:20:01.915941 23581 target.go:112] Terraform output is in /tmp/test991935992/out I1007 22:20:01.920409 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:01.920642 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:01.920701 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:01.920754 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:01.920808 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:01.920851 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:01.920880 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:01.920915 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:01.920952 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:01.943892 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:01.946825 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:01.946915 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:01.988464 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:02.086018 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:02.086475 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:02.086861 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:02.086893 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:02.086950 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:02.086909 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:02.087044 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:02.087012 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:02.087097 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:02.087270 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1007 22:20:02.088845 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:02.093942 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:02.094268 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:02.094454 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:02.095300 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:02.096537 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:02.096674 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:02.096892 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:02.097091 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:02.099170 23581 target.go:168] Cloudformation output is in /tmp/test2970054693/out --- PASS: TestMinimal_v1_23 (1.73s) === RUN TestMinimal_v1_24 I1007 22:20:02.111656 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:02.111912 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:02.111965 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:02.112055 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:02.112109 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:02.112152 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:02.112186 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:02.112229 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:02.112259 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:02.138939 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:02.142549 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:02.142643 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.096500 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:03.276116 23581 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1007 22:20:03.276536 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:03.276775 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:03.276788 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:03.276809 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:03.276913 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:03.276589 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:03.283750 23581 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1007 22:20:03.289081 23581 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1007 22:20:03.289616 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.289856 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.290847 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.292501 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.292628 23581 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1007 22:20:03.292889 23581 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1007 22:20:03.293043 23581 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1007 22:20:03.368704 23581 target.go:112] Terraform output is in /tmp/test3908691059/out I1007 22:20:03.373281 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:03.373484 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:03.373543 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:03.373592 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:03.373644 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:03.373708 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:03.373740 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:03.373777 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:03.373804 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:03.395208 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:03.398204 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.398295 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:03.438759 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:03.523367 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:03.523701 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:03.523816 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:03.523904 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:03.523937 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:03.523849 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:03.525018 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:03.524129 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } W1007 22:20:03.525068 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:03.525928 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:03.529605 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:03.535717 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:03.536046 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.536177 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.536763 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.538838 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.538975 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:03.539251 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:03.539436 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:03.542311 23581 target.go:168] Cloudformation output is in /tmp/test3963938269/out --- PASS: TestMinimal_v1_24 (1.44s) === RUN TestMinimal_v1_25 I1007 22:20:03.551983 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:03.552202 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:03.552253 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:03.552341 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:03.552392 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:03.552433 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:03.552462 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:03.552504 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:03.552532 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:03.573510 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:03.577846 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:03.577952 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.011889 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:05.191660 23581 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1007 22:20:05.192065 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:05.192152 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:05.192094 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:05.192197 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:05.192383 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:05.192510 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:05.194428 23581 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1007 22:20:05.199501 23581 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1007 22:20:05.199919 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.201810 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.204962 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.206519 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.206651 23581 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1007 22:20:05.206983 23581 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1007 22:20:05.207251 23581 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1007 22:20:05.262657 23581 target.go:112] Terraform output is in /tmp/test2706314188/out I1007 22:20:05.269804 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:05.270054 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:05.270121 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:05.270179 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:05.270239 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:05.270296 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:05.270326 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:05.270365 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:05.270473 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:05.294949 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:05.298183 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.298287 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:05.339063 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:05.444676 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:05.444953 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:05.444991 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:05.445116 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:05.445483 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:05.445719 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:05.445820 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:05.446031 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:05.446062 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:05.446194 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1007 22:20:05.447216 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:05.452713 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:05.452925 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.453295 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.455053 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.457542 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.457642 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:05.457864 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:05.458034 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:05.459683 23581 target.go:168] Cloudformation output is in /tmp/test61708773/out --- PASS: TestMinimal_v1_25 (1.91s) === RUN TestMinimal_v1_26 I1007 22:20:05.465061 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:05.465195 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:05.465230 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:05.465260 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:05.465292 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:05.465345 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:05.465362 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:05.465385 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:05.465402 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:05.484612 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* This version of kubernetes is not yet supported; upgrading kops is required (you can bypass this check by exporting KOPS_RUN_TOO_NEW_VERSION) ********************************************************************************* I1007 22:20:05.488605 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:05.488701 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:06.949524 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:07.122621 23581 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1007 22:20:07.123291 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:07.123350 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:07.123326 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:07.123414 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:07.123653 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:07.123775 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:07.129271 23581 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1007 22:20:07.136035 23581 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1007 22:20:07.136386 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.136646 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.138348 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.140800 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.140935 23581 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1007 22:20:07.141297 23581 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1007 22:20:07.141524 23581 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1007 22:20:07.194566 23581 target.go:112] Terraform output is in /tmp/test2830981188/out I1007 22:20:07.198675 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:07.198898 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:07.198962 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:07.199018 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:07.199075 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:07.199144 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:07.199174 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:07.199213 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:07.199244 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:07.226508 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:07.229803 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.229910 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:07.270950 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:07.373077 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:07.373336 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:07.373402 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:07.373596 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:07.373628 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:07.373953 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:07.373959 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:07.374010 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:07.373842 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:07.374151 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:07.376670 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:07.381965 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:07.382309 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.382550 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.383651 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.384585 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:07.384722 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:07.384991 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:07.385180 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:07.388084 23581 target.go:168] Cloudformation output is in /tmp/test338083350/out --- PASS: TestMinimal_v1_26 (1.94s) === RUN TestHetzner I1007 22:20:07.401818 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:08.402203 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation I1007 22:20:08.528091 23581 executor.go:111] Tasks: 0 done / 50 total; 39 can run I1007 22:20:08.534869 23581 executor.go:111] Tasks: 39 done / 50 total; 5 can run I1007 22:20:08.536423 23581 executor.go:111] Tasks: 44 done / 50 total; 2 can run I1007 22:20:08.541671 23581 executor.go:111] Tasks: 46 done / 50 total; 4 can run I1007 22:20:08.544497 23581 executor.go:111] Tasks: 50 done / 50 total; 0 can run I1007 22:20:08.562501 23581 target.go:112] Terraform output is in /tmp/test178964744/out --- PASS: TestHetzner (1.16s) === RUN TestNvidia I1007 22:20:08.566039 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:08.566270 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:08.566325 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:08.566437 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:08.566488 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:08.566529 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:08.566561 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:08.566599 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:08.566626 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:08.590716 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:08.594008 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.594104 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.635593 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:08.742000 23581 executor.go:111] Tasks: 0 done / 88 total; 44 can run W1007 22:20:08.742429 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:08.742729 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:08.742721 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:08.742767 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:08.742874 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:08.743165 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:08.745043 23581 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1007 22:20:08.750211 23581 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1007 22:20:08.750432 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.750790 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.752961 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.756804 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.756949 23581 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1007 22:20:08.757364 23581 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1007 22:20:08.757647 23581 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1007 22:20:08.807317 23581 target.go:112] Terraform output is in /tmp/test2512272648/out I1007 22:20:08.811284 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:08.811492 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:08.811551 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:08.811607 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:08.811662 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:08.811704 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:08.811734 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:08.811773 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:08.811802 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:08.839666 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:08.843067 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.843172 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:08.884129 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:08.991593 23581 executor.go:111] Tasks: 0 done / 88 total; 44 can run I1007 22:20:08.992161 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:08.992194 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:08.992308 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:08.992424 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:08.992402 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:08.992590 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1007 22:20:08.992647 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:08.992623 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:08.992805 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1007 22:20:08.994973 23581 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1007 22:20:09.003930 23581 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1007 22:20:09.004157 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.004604 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.005491 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.007485 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.007622 23581 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1007 22:20:09.007888 23581 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1007 22:20:09.008084 23581 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1007 22:20:09.011289 23581 target.go:168] Cloudformation output is in /tmp/test1910010728/out --- PASS: TestNvidia (0.46s) === RUN TestMinimalGossip I1007 22:20:09.021609 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:09.021860 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:09.021915 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:09.021963 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:09.022018 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:09.022105 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:09.022136 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:09.022177 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:09.022207 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:09.049233 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.049331 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.090112 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation I1007 22:20:09.193042 23581 executor.go:111] Tasks: 0 done / 86 total; 42 can run I1007 22:20:09.193640 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:09.193905 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:09.194047 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:09.197852 23581 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1007 22:20:09.205808 23581 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1007 22:20:09.206013 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.206393 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.208118 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.209206 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.209338 23581 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1007 22:20:09.209834 23581 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1007 22:20:09.210145 23581 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1007 22:20:09.264933 23581 target.go:112] Terraform output is in /tmp/test696962250/out --- PASS: TestMinimalGossip (0.25s) === RUN TestMinimalGossipIRSA I1007 22:20:09.268941 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:09.269177 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:09.269237 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:09.269289 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:09.269344 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:09.269437 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:09.269469 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:09.269509 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:09.269539 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:09.297613 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.297711 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.339870 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation I1007 22:20:09.449735 23581 executor.go:111] Tasks: 0 done / 89 total; 44 can run W1007 22:20:09.450239 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:09.450540 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:09.450766 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:09.457089 23581 executor.go:111] Tasks: 44 done / 89 total; 20 can run I1007 22:20:09.462680 23581 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1007 22:20:09.463087 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.463289 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.464222 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.466060 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:09.466177 23581 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1007 22:20:09.466477 23581 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1007 22:20:09.466660 23581 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1007 22:20:09.522389 23581 target.go:112] Terraform output is in /tmp/test3634641841/out --- PASS: TestMinimalGossipIRSA (0.26s) === RUN TestMinimalGCE I1007 22:20:09.526941 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:09.528886 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1007 22:20:10.452601 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:10.452702 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:10.452844 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1007 22:20:10.452888 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1007 22:20:10.452914 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:10.452947 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:10.453025 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:10.454800 23581 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1007 22:20:10.457511 23581 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1007 22:20:10.466439 23581 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1007 22:20:10.469247 23581 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1007 22:20:10.469749 23581 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1007 22:20:10.502442 23581 target.go:112] Terraform output is in /tmp/test3428923995/out --- PASS: TestMinimalGCE (0.98s) === RUN TestMinimalGCEPrivate I1007 22:20:10.506296 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:10.508116 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1007 22:20:11.408501 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:11.408593 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:11.408719 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1007 22:20:11.408783 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W1007 22:20:11.408810 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W1007 22:20:11.408835 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1007 22:20:11.408900 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:11.410793 23581 executor.go:111] Tasks: 0 done / 66 total; 40 can run I1007 22:20:11.414065 23581 executor.go:111] Tasks: 40 done / 66 total; 18 can run I1007 22:20:11.419962 23581 executor.go:111] Tasks: 58 done / 66 total; 4 can run I1007 22:20:11.425722 23581 executor.go:111] Tasks: 62 done / 66 total; 3 can run I1007 22:20:11.427248 23581 executor.go:111] Tasks: 65 done / 66 total; 1 can run I1007 22:20:11.427413 23581 executor.go:111] Tasks: 66 done / 66 total; 0 can run I1007 22:20:11.460740 23581 target.go:112] Terraform output is in /tmp/test1770513607/out --- PASS: TestMinimalGCEPrivate (0.96s) === RUN TestMinimalGCEInternalLoadBalancer I1007 22:20:11.464431 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:11.466280 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1007 22:20:11.643813 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:11.643897 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:11.644016 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W1007 22:20:11.644052 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W1007 22:20:11.644082 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1007 22:20:11.644116 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1007 22:20:11.644187 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1007 22:20:11.644232 23581 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1007 22:20:11.645805 23581 executor.go:111] Tasks: 0 done / 68 total; 41 can run I1007 22:20:11.649191 23581 executor.go:111] Tasks: 41 done / 68 total; 17 can run W1007 22:20:11.654600 23581 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1007 22:20:11.654639 23581 executor.go:111] Tasks: 58 done / 68 total; 4 can run I1007 22:20:11.657407 23581 executor.go:111] Tasks: 62 done / 68 total; 3 can run I1007 22:20:11.658736 23581 executor.go:111] Tasks: 65 done / 68 total; 2 can run I1007 22:20:11.659138 23581 executor.go:111] Tasks: 67 done / 68 total; 1 can run I1007 22:20:11.659404 23581 executor.go:111] Tasks: 68 done / 68 total; 0 can run I1007 22:20:11.693675 23581 target.go:112] Terraform output is in /tmp/test506441613/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.23s) === RUN TestMinimalGCELongClusterName I1007 22:20:11.697693 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:11.699665 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1007 22:20:11.936452 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:11.936536 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:11.936629 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1007 22:20:11.936680 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1007 22:20:11.936702 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1007 22:20:11.936722 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1007 22:20:11.936780 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:11.938156 23581 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1007 22:20:11.945729 23581 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1007 22:20:11.951659 23581 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1007 22:20:11.956042 23581 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1007 22:20:11.956384 23581 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1007 22:20:11.994050 23581 target.go:112] Terraform output is in /tmp/test3392569937/out --- PASS: TestMinimalGCELongClusterName (0.30s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I1007 22:20:11.998189 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:12.000132 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1007 22:20:12.225866 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:12.225956 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:12.226101 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1007 22:20:12.226137 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1007 22:20:12.226165 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1007 22:20:12.226199 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1007 22:20:12.226265 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1007 22:20:12.226315 23581 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1007 22:20:12.228002 23581 executor.go:111] Tasks: 0 done / 67 total; 42 can run I1007 22:20:12.235606 23581 executor.go:111] Tasks: 42 done / 67 total; 16 can run W1007 22:20:12.241927 23581 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1007 22:20:12.241978 23581 executor.go:111] Tasks: 58 done / 67 total; 5 can run I1007 22:20:12.245305 23581 executor.go:111] Tasks: 63 done / 67 total; 2 can run I1007 22:20:12.245480 23581 executor.go:111] Tasks: 65 done / 67 total; 1 can run I1007 22:20:12.245651 23581 executor.go:111] Tasks: 66 done / 67 total; 1 can run I1007 22:20:12.245790 23581 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1007 22:20:12.278225 23581 target.go:112] Terraform output is in /tmp/test480237084/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.28s) === RUN TestHA I1007 22:20:12.282329 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:12.282570 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:12.282631 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:12.282685 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:12.282741 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:12.282853 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:12.282885 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:12.282924 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:12.282954 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:12.315243 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:12.318700 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.318785 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.318835 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.318888 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.360272 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:12.475371 23581 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1007 22:20:12.476131 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:12.476138 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:12.476398 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:12.476480 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:12.476492 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:12.476563 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:12.476606 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:12.476764 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:12.478286 23581 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1007 22:20:12.484588 23581 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1007 22:20:12.484873 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.485036 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.485189 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.487009 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.488279 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.492185 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.492645 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.492769 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.492917 23581 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1007 22:20:12.493365 23581 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1007 22:20:12.493655 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:12.568983 23581 target.go:112] Terraform output is in /tmp/test4061184722/out --- PASS: TestHA (0.29s) === RUN TestHighAvailabilityGCE I1007 22:20:12.574222 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:12.576341 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1007 22:20:12.770656 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:12.770759 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:12.770885 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W1007 22:20:12.770921 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W1007 22:20:12.770948 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1007 22:20:12.770982 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1007 22:20:12.771047 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1007 22:20:12.771129 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1007 22:20:12.771200 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:12.773341 23581 executor.go:111] Tasks: 0 done / 83 total; 48 can run I1007 22:20:12.776545 23581 executor.go:111] Tasks: 48 done / 83 total; 20 can run I1007 22:20:12.785692 23581 executor.go:111] Tasks: 68 done / 83 total; 7 can run I1007 22:20:12.790229 23581 executor.go:111] Tasks: 75 done / 83 total; 5 can run I1007 22:20:12.791835 23581 executor.go:111] Tasks: 80 done / 83 total; 3 can run I1007 22:20:12.792102 23581 executor.go:111] Tasks: 83 done / 83 total; 0 can run I1007 22:20:12.833251 23581 target.go:112] Terraform output is in /tmp/test466839524/out --- PASS: TestHighAvailabilityGCE (0.26s) === RUN TestComplex I1007 22:20:12.838598 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:12.838828 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:12.838888 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:12.839016 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:12.839071 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:12.839113 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:12.839144 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:12.839182 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:12.839210 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:12.876273 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:12.880916 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:12.881032 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:13.914256 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:14.082791 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1007 22:20:14.083512 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:14.083630 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:14.083561 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:14.083693 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:14.083943 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:14.089609 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:14.096035 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:14.096509 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.096790 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.098455 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.099827 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.099951 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:14.100732 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:14.101068 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:14.175993 23581 target.go:112] Terraform output is in /tmp/test372913998/out I1007 22:20:14.180576 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:14.180798 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:14.180853 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:14.180904 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:14.180958 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:14.181002 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:14.181031 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:14.181069 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:14.181098 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:14.213474 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:14.219537 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.219872 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.262386 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:14.456646 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1007 22:20:14.457485 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:14.457525 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:14.457625 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:14.457594 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:14.457715 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:14.459543 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:14.465277 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:14.465998 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.467314 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.471099 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.472825 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.472979 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:14.473324 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:14.473594 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:14.478677 23581 target.go:168] Cloudformation output is in /tmp/test4164486310/out I1007 22:20:14.492520 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:14.492738 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:14.492792 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:14.492840 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:14.492892 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:14.492935 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:14.492965 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:14.493005 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:14.493035 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:14.524278 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:14.527962 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.528048 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.568803 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:14.767255 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1007 22:20:14.768268 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:14.768297 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:14.768371 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:14.768342 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:14.768490 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:14.770210 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:14.780382 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:14.781192 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.781429 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.782603 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.784335 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.784468 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:14.784856 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:14.785043 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:14.856911 23581 target.go:112] Terraform output is in /tmp/test2190872725/out --- PASS: TestComplex (2.02s) === RUN TestCompress I1007 22:20:14.861760 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:14.862006 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:14.862070 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:14.862184 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:14.862241 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:14.862289 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:14.862319 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:14.862359 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:14.862387 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:14.890868 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:14.894126 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.894232 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:14.934888 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:15.033051 23581 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1007 22:20:15.033422 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:15.033421 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:15.033645 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:15.033687 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:15.033791 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:15.036147 23581 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1007 22:20:15.042748 23581 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1007 22:20:15.043051 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.043354 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.046313 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.052201 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.052402 23581 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1007 22:20:15.052856 23581 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1007 22:20:15.053257 23581 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1007 22:20:15.099600 23581 target.go:112] Terraform output is in /tmp/test1570774991/out --- PASS: TestCompress (0.24s) === RUN TestExternalPolicies I1007 22:20:15.103419 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:15.103661 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:15.103721 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:15.103775 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:15.103853 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:15.103969 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:15.104010 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:15.104053 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:15.104083 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:15.128454 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:15.132229 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.132325 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.173570 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:15.269310 23581 executor.go:111] Tasks: 0 done / 100 total; 45 can run W1007 22:20:15.270855 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:15.271825 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:15.273755 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:15.273793 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:15.274409 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:15.274585 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:15.276714 23581 executor.go:111] Tasks: 45 done / 100 total; 20 can run I1007 22:20:15.282173 23581 executor.go:111] Tasks: 65 done / 100 total; 30 can run I1007 22:20:15.282615 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.282893 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.284187 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.285038 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.285158 23581 executor.go:111] Tasks: 95 done / 100 total; 3 can run I1007 22:20:15.285731 23581 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1007 22:20:15.285947 23581 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1007 22:20:15.355064 23581 target.go:112] Terraform output is in /tmp/test1986530821/out --- PASS: TestExternalPolicies (0.26s) === RUN TestMinimalIPv6 I1007 22:20:15.359112 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:15.359347 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:15.359404 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:15.359509 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:15.359561 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:15.359603 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:15.359632 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:15.359669 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:15.359699 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:15.385286 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:15.388838 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.388922 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.431037 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:15.605312 23581 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1007 22:20:15.605798 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:15.605996 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:15.606045 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:15.606038 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:15.606161 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:15.606183 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:15.608293 23581 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1007 22:20:15.614367 23581 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1007 22:20:15.614919 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.615043 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.616374 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.618000 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.618224 23581 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1007 22:20:15.618664 23581 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1007 22:20:15.619121 23581 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1007 22:20:15.679490 23581 target.go:112] Terraform output is in /tmp/test309392780/out I1007 22:20:15.683894 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:15.684113 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:15.684193 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:15.684253 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:15.684310 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:15.684354 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:15.684387 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:15.684429 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:15.684456 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:15.712999 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:15.716819 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.716922 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.757349 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:15.925670 23581 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1007 22:20:15.926159 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:15.926401 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:15.926594 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:15.926660 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:15.926691 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:15.926794 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:15.926809 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:15.926822 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:15.926966 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:15.929217 23581 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1007 22:20:15.934634 23581 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1007 22:20:15.935112 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.935278 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.936477 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.938822 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.939190 23581 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1007 22:20:15.939886 23581 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1007 22:20:15.940848 23581 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1007 22:20:15.946306 23581 target.go:168] Cloudformation output is in /tmp/test3992921411/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I1007 22:20:15.958703 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:15.958932 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:15.958985 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:15.959036 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:15.959089 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:15.959169 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:15.959198 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:15.959237 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:15.959266 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:15.988245 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1007 22:20:15.991929 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:15.992011 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.032346 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:16.208681 23581 executor.go:111] Tasks: 0 done / 124 total; 46 can run I1007 22:20:16.209329 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:16.209342 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:16.209368 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:16.209463 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:16.209499 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:16.209590 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:16.216017 23581 executor.go:111] Tasks: 46 done / 124 total; 23 can run I1007 22:20:16.221574 23581 executor.go:111] Tasks: 69 done / 124 total; 36 can run I1007 22:20:16.222267 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.222495 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.223725 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.225314 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.225533 23581 executor.go:111] Tasks: 105 done / 124 total; 11 can run I1007 22:20:16.226064 23581 executor.go:111] Tasks: 116 done / 124 total; 8 can run I1007 22:20:16.226497 23581 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1007 22:20:16.303438 23581 target.go:112] Terraform output is in /tmp/test1016151970/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I1007 22:20:16.308055 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:16.308290 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:16.308348 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:16.308460 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:16.308515 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:16.308558 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:16.308587 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:16.308627 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:16.308658 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:16.334633 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:16.338334 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.338426 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.381895 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:16.839679 23581 executor.go:111] Tasks: 0 done / 109 total; 47 can run I1007 22:20:16.839973 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:16.840232 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:16.840447 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:16.840617 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:16.840650 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:16.840754 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:16.842561 23581 executor.go:111] Tasks: 47 done / 109 total; 21 can run I1007 22:20:16.847690 23581 executor.go:111] Tasks: 68 done / 109 total; 31 can run I1007 22:20:16.848141 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.848316 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.850600 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.854640 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.854814 23581 executor.go:111] Tasks: 99 done / 109 total; 5 can run I1007 22:20:16.855365 23581 executor.go:111] Tasks: 104 done / 109 total; 5 can run I1007 22:20:16.855728 23581 executor.go:111] Tasks: 109 done / 109 total; 0 can run I1007 22:20:16.922002 23581 target.go:112] Terraform output is in /tmp/test4237856498/out --- PASS: TestMinimalIPv6Calico (0.62s) === RUN TestMinimalIPv6Cilium I1007 22:20:16.927506 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:16.927800 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:16.927869 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:16.928007 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:16.928082 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:16.928140 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:16.928184 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:16.928239 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:16.928276 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:16.954235 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:16.958385 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:16.958490 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.000288 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:17.207753 23581 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1007 22:20:17.210301 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:17.210588 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:17.210750 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:17.211010 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:17.211050 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:17.211144 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:17.215116 23581 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1007 22:20:17.220207 23581 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1007 22:20:17.220687 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.220971 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.221953 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.224155 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.224299 23581 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1007 22:20:17.224707 23581 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1007 22:20:17.225000 23581 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1007 22:20:17.291443 23581 target.go:112] Terraform output is in /tmp/test1690801124/out I1007 22:20:17.296023 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:17.296231 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:17.296289 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:17.296343 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:17.296399 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:17.296442 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:17.296473 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:17.296512 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:17.296540 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:17.321715 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:17.325726 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.325822 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.366240 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:17.582468 23581 executor.go:111] Tasks: 0 done / 107 total; 46 can run I1007 22:20:17.582904 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:17.582939 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:17.583054 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:17.583033 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:17.584379 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:17.584406 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:17.584524 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } W1007 22:20:17.585036 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:17.585253 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:17.590299 23581 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1007 22:20:17.595625 23581 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1007 22:20:17.596054 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.596278 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.597561 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.598810 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.598967 23581 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1007 22:20:17.599544 23581 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1007 22:20:17.599922 23581 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1007 22:20:17.603445 23581 target.go:168] Cloudformation output is in /tmp/test1591852099/out --- PASS: TestMinimalIPv6Cilium (0.69s) === RUN TestMinimalWarmPool I1007 22:20:17.615928 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:17.616307 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:17.616424 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:17.616541 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:17.616660 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:17.616823 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:17.616911 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:17.617008 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:17.617073 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:17.644361 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:17.648220 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.648327 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.690531 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:17.894651 23581 executor.go:111] Tasks: 0 done / 89 total; 45 can run W1007 22:20:17.895209 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:17.895240 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:17.895280 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:17.895422 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:17.895433 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:17.895506 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:17.897562 23581 executor.go:111] Tasks: 45 done / 89 total; 19 can run I1007 22:20:17.902909 23581 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1007 22:20:17.903274 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.903528 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.905889 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.909853 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.910005 23581 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1007 22:20:17.910359 23581 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1007 22:20:17.910574 23581 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1007 22:20:17.959683 23581 target.go:112] Terraform output is in /tmp/test3481673423/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I1007 22:20:17.964283 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:17.964528 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:17.964590 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:17.964698 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:17.964754 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:17.964801 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:17.964829 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:17.964870 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:17.964903 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:17.995109 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:17.998520 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:17.998618 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.038921 23581 zones.go:93] ListHostedZonesPages { } W1007 22:20:18.120336 23581 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W1007 22:20:18.120392 23581 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W1007 22:20:18.122184 23581 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:18.137982 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:18.138567 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:18.138557 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:18.138695 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1007 22:20:18.138782 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:18.138716 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:18.138981 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:18.138878 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I1007 22:20:18.139035 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:18.139171 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:18.141495 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:18.146652 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:18.146988 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.147279 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.148134 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.149930 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.150066 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:18.150340 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:18.150574 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:18.153514 23581 target.go:168] Cloudformation output is in /tmp/test2619454684/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I1007 22:20:18.163317 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:18.163540 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:18.163591 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:18.163637 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:18.163687 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:18.163763 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:18.163792 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:18.163847 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:18.163878 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:18.193501 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:18.196795 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.196892 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.237888 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:18.343447 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:18.344090 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:18.344414 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:18.344563 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:18.344547 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:18.344624 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:18.344785 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:18.347111 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:18.354274 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:18.354546 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.354753 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.355985 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.357053 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.357184 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:18.357492 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:18.357689 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:18.409812 23581 target.go:112] Terraform output is in /tmp/test3623304814/out I1007 22:20:18.413741 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:18.413980 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:18.414042 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:18.414095 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:18.414153 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:18.414201 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:18.414232 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:18.414274 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:18.414304 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:18.439496 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:18.442909 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.443011 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.484569 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:18.586129 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:18.586432 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:18.586549 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:18.586578 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:18.586670 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:18.586779 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:18.586911 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:18.586929 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:18.587026 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } W1007 22:20:18.586907 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:18.588912 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:18.594082 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:18.594544 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.594745 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.595828 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.596971 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.597104 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:18.597348 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:18.597615 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:18.601890 23581 target.go:168] Cloudformation output is in /tmp/test991274344/out --- PASS: TestMinimalGp3 (0.45s) === RUN TestMinimalLongClusterName I1007 22:20:18.613061 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:18.613294 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:18.613349 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:18.613487 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:18.613705 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:18.613791 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:18.613883 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:18.613956 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:18.614010 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:18.638655 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:18.641911 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.642016 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:18.682817 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:18.785218 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:18.785826 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:18.785896 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:18.786254 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:18.786309 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:18.786433 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:18.786559 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:18.788797 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:18.794101 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:18.794333 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.794475 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.795411 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.796759 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.796875 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:18.797235 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:18.797492 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:18.854753 23581 target.go:112] Terraform output is in /tmp/test1005198893/out I1007 22:20:18.858819 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:18.859035 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:18.859098 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:18.859151 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:18.859208 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:18.859252 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:18.859284 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:18.859322 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:18.859354 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:18.883447 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:18.886788 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:18.886883 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:18.928135 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:19.028687 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:19.029173 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:19.029328 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:19.029348 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:19.031442 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:19.029457 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } W1007 22:20:19.032097 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:19.032470 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:19.032500 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:19.032594 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:19.036213 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:19.041796 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:19.042041 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.042350 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.043557 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.044399 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.044539 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:19.044838 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:19.045215 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:19.048727 23581 target.go:168] Cloudformation output is in /tmp/test284467133/out --- PASS: TestMinimalLongClusterName (0.45s) === RUN TestExistingIAMCloudformation I1007 22:20:19.058740 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:19.058961 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:19.059013 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:19.059102 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:19.059157 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:19.059200 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:19.059229 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:19.059269 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:19.059295 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:19.087269 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:19.090838 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.090936 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.132084 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:19.233503 23581 executor.go:111] Tasks: 0 done / 77 total; 41 can run W1007 22:20:19.234148 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:19.234281 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:19.234358 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:19.234393 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:19.234487 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:19.234511 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:19.234483 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:19.234554 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:19.234681 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:19.238025 23581 executor.go:111] Tasks: 41 done / 77 total; 11 can run I1007 22:20:19.245674 23581 executor.go:111] Tasks: 52 done / 77 total; 21 can run I1007 22:20:19.246125 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.246364 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.247682 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.248995 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.249140 23581 executor.go:111] Tasks: 73 done / 77 total; 2 can run I1007 22:20:19.249412 23581 executor.go:111] Tasks: 75 done / 77 total; 2 can run I1007 22:20:19.249614 23581 executor.go:111] Tasks: 77 done / 77 total; 0 can run I1007 22:20:19.252239 23581 target.go:168] Cloudformation output is in /tmp/test1931587346/out --- PASS: TestExistingIAMCloudformation (0.20s) === RUN TestExistingSG I1007 22:20:19.260265 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:19.260509 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:19.260567 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:19.260616 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:19.260669 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:19.260752 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:19.260782 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:19.260818 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:19.260847 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:19.296925 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:19.300983 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.301083 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.301146 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.301205 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.341788 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:19.457803 23581 executor.go:111] Tasks: 0 done / 146 total; 53 can run I1007 22:20:19.458324 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:19.458361 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:19.458455 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:19.458469 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:19.458569 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:19.458820 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:19.458917 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:19.460410 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:19.465138 23581 executor.go:111] Tasks: 53 done / 146 total; 26 can run I1007 22:20:19.470705 23581 executor.go:111] Tasks: 79 done / 146 total; 58 can run I1007 22:20:19.471138 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.471334 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.471564 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.471892 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.472413 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.473777 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.475423 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.475566 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.475727 23581 executor.go:111] Tasks: 137 done / 146 total; 5 can run I1007 22:20:19.476537 23581 executor.go:111] Tasks: 142 done / 146 total; 4 can run I1007 22:20:19.477068 23581 executor.go:111] Tasks: 146 done / 146 total; 0 can run I1007 22:20:19.574462 23581 target.go:112] Terraform output is in /tmp/test3795845242/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I1007 22:20:19.580051 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:19.580291 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:19.580356 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:19.580469 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:19.580543 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:19.580590 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:19.580619 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:19.580658 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:19.580685 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:19.614160 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:19.618517 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.618695 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.618815 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.661585 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:19.766627 23581 executor.go:111] Tasks: 0 done / 122 total; 48 can run I1007 22:20:19.767299 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:19.767386 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:19.767444 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:19.767488 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:19.767417 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:19.767552 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:19.767559 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:19.773666 23581 executor.go:111] Tasks: 48 done / 122 total; 29 can run I1007 22:20:19.779302 23581 executor.go:111] Tasks: 77 done / 122 total; 36 can run I1007 22:20:19.779631 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.779822 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.780078 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.780863 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.781342 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.782241 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.782373 23581 executor.go:111] Tasks: 113 done / 122 total; 5 can run I1007 22:20:19.782766 23581 executor.go:111] Tasks: 118 done / 122 total; 4 can run I1007 22:20:19.783071 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:19.861922 23581 target.go:112] Terraform output is in /tmp/test2848792274/out --- PASS: TestBastionAdditionalUserData (0.29s) === RUN TestPrivateWeave I1007 22:20:19.866476 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:19.866718 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:19.866777 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:19.866832 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:19.866890 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:19.866984 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:19.867014 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:19.867055 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:19.867087 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:19.898355 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:19.902128 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.902236 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.902327 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:19.943311 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:20.068411 23581 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1007 22:20:20.068830 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:20.069016 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:20.069041 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:20.069128 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:20.069172 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:20.069222 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:20.069217 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:20.070643 23581 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1007 22:20:20.078002 23581 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1007 22:20:20.078289 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.078606 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.078834 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.078931 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.079514 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.081284 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.081420 23581 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1007 22:20:20.081977 23581 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1007 22:20:20.082284 23581 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1007 22:20:20.156605 23581 target.go:112] Terraform output is in /tmp/test2323442770/out --- PASS: TestPrivateWeave (0.29s) === RUN TestPrivateFlannel I1007 22:20:20.161021 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:20.161259 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:20.161321 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:20.161431 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:20.161487 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:20.161534 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:20.161563 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:20.161602 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:20.161632 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:20.191438 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:20.195365 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.195465 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.195528 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.235550 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:20.431529 23581 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1007 22:20:20.432039 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:20.432088 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:20.432153 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:20.432498 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:20.432566 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:20.432607 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:20.432752 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:20.436315 23581 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1007 22:20:20.444866 23581 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1007 22:20:20.445297 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.445503 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.445729 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.445794 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.447097 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.448328 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.448601 23581 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1007 22:20:20.449136 23581 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1007 22:20:20.449470 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:20.526387 23581 target.go:112] Terraform output is in /tmp/test1470259484/out --- PASS: TestPrivateFlannel (0.37s) === RUN TestPrivateCalico I1007 22:20:20.531323 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:20.531556 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:20.531619 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:20.531734 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:20.531786 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:20.531831 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:20.531863 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:20.531903 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:20.531933 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:20.561055 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:20.564823 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.564914 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.564975 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:20.605718 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:21.070861 23581 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1007 22:20:21.071482 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:21.071734 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:21.071798 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:21.071838 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:21.071971 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:21.071802 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:21.071999 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:21.073700 23581 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1007 22:20:21.079293 23581 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1007 22:20:21.079683 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.080012 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.080150 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.081378 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.083334 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.087088 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.087246 23581 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1007 22:20:21.087881 23581 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1007 22:20:21.088341 23581 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1007 22:20:21.164172 23581 target.go:112] Terraform output is in /tmp/test111104003/out --- PASS: TestPrivateCalico (0.64s) === RUN TestPrivateCilium I1007 22:20:21.169516 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:21.169757 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:21.169824 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:21.169934 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:21.169993 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:21.170039 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:21.170069 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:21.170106 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:21.170135 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:21.198689 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:21.202305 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.202392 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.202486 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.243363 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:21.395123 23581 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1007 22:20:21.395797 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:21.395782 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:21.396012 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:21.396137 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:21.396056 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:21.396217 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:21.396319 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:21.398044 23581 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1007 22:20:21.407106 23581 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1007 22:20:21.407774 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.407992 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.408233 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.408510 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.409421 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.410545 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.410760 23581 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1007 22:20:21.411334 23581 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1007 22:20:21.411668 23581 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1007 22:20:21.487431 23581 target.go:112] Terraform output is in /tmp/test2276444264/out I1007 22:20:21.492081 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:21.492361 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:21.492434 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:21.492493 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:21.492562 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:21.492618 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:21.492654 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:21.492708 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:21.492740 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:21.524318 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:21.529020 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.529138 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.529225 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.570736 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:21.720450 23581 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1007 22:20:21.721023 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:21.721010 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:21.721150 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1007 22:20:21.721232 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:21.721167 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:21.721245 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:21.721483 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:21.721507 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:21.722364 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:21.721314 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1007 22:20:21.728401 23581 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1007 22:20:21.734227 23581 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1007 22:20:21.734686 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.734976 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.735253 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.735428 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.736351 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.738101 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.738248 23581 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1007 22:20:21.738590 23581 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1007 22:20:21.738915 23581 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1007 22:20:21.743296 23581 target.go:168] Cloudformation output is in /tmp/test4053423038/out --- PASS: TestPrivateCilium (0.59s) === RUN TestPrivateCilium2 I1007 22:20:21.756319 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:21.756540 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:21.756600 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:21.756689 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:21.756746 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:21.756790 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:21.756819 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:21.756860 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:21.756890 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:21.789921 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:21.794022 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.794126 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.794190 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:21.834703 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:22.491398 23581 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1007 22:20:22.491774 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:22.491974 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:22.492241 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:22.492434 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:22.492509 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:22.492537 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:22.492643 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:22.496791 23581 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1007 22:20:22.503254 23581 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1007 22:20:22.504156 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.504793 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.505029 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.505711 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.508887 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.510223 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.510365 23581 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1007 22:20:22.511010 23581 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1007 22:20:22.511283 23581 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1007 22:20:22.583062 23581 target.go:112] Terraform output is in /tmp/test1775120658/out I1007 22:20:22.590355 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:22.590613 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:22.590675 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:22.590730 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:22.590783 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:22.590826 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:22.590855 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:22.590894 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:22.590922 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:22.624701 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:22.629373 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.629483 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.629551 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:22.671068 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:23.336142 23581 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1007 22:20:23.336761 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:23.336933 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:23.337022 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:23.337045 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:23.337043 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:23.337194 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:23.337206 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:23.337389 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:23.337420 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:23.339044 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1007 22:20:23.345657 23581 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1007 22:20:23.351992 23581 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1007 22:20:23.352503 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.352721 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.353001 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.353175 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.354084 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.355349 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.355492 23581 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1007 22:20:23.355882 23581 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1007 22:20:23.356131 23581 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1007 22:20:23.360267 23581 target.go:168] Cloudformation output is in /tmp/test42957790/out --- PASS: TestPrivateCilium2 (1.62s) === RUN TestPrivateCiliumAdvanced I1007 22:20:23.376600 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:23.376875 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:23.376926 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:23.377033 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:23.377087 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:23.377132 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:23.377182 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:23.377225 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:23.377255 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:23.411068 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:23.415093 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.415194 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.415259 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.456333 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:23.613484 23581 executor.go:111] Tasks: 0 done / 126 total; 53 can run W1007 22:20:23.614384 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:23.615741 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:23.614444 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:23.617141 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:23.617208 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:23.617372 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:23.618519 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:23.622590 23581 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1007 22:20:23.629297 23581 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1007 22:20:23.629501 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.630014 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.630245 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.630462 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.631304 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.631971 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.632097 23581 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1007 22:20:23.632549 23581 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1007 22:20:23.632816 23581 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1007 22:20:23.714776 23581 target.go:112] Terraform output is in /tmp/test807902611/out I1007 22:20:23.719625 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:23.719883 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:23.719962 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:23.720046 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:23.720140 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:23.720205 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:23.720267 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:23.720334 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:23.720382 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:23.754000 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:23.761051 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.761161 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.761255 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.802734 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:23.955867 23581 executor.go:111] Tasks: 0 done / 126 total; 53 can run I1007 22:20:23.956367 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:23.956413 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:23.956510 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:23.956536 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:23.958229 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:23.958425 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:23.958446 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:23.958576 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } W1007 22:20:23.958862 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:23.960433 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:23.964612 23581 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1007 22:20:23.971183 23581 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1007 22:20:23.971638 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.971912 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.972069 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.972130 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.972759 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.974808 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:23.974960 23581 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1007 22:20:23.975343 23581 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1007 22:20:23.975616 23581 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1007 22:20:23.979714 23581 target.go:168] Cloudformation output is in /tmp/test1919361588/out --- PASS: TestPrivateCiliumAdvanced (0.62s) === RUN TestPrivateCanal I1007 22:20:23.997264 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:23.997503 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:23.997562 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:23.997669 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:23.997722 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:23.997767 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:23.997797 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:23.997838 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:23.997867 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:24.026376 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:24.029942 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.030050 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.030114 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.070884 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:24.540120 23581 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1007 22:20:24.542067 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:24.542424 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:24.540500 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:24.543102 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:24.543129 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:24.543232 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:24.544244 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:24.548493 23581 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1007 22:20:24.554771 23581 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1007 22:20:24.555305 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.555580 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.555810 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.555952 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.557412 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.558356 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:24.558480 23581 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1007 22:20:24.559924 23581 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1007 22:20:24.560440 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:24.635254 23581 target.go:112] Terraform output is in /tmp/test2500422900/out --- PASS: TestPrivateCanal (0.64s) === RUN TestPrivateKopeio I1007 22:20:24.640933 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:24.641176 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:24.641238 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:24.641291 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:24.641346 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:24.641449 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:24.641479 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:24.641518 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:24.641549 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:24.668372 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:24.671761 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.671851 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.671907 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.712743 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:24.832397 23581 executor.go:111] Tasks: 0 done / 126 total; 47 can run W1007 22:20:24.832791 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:24.833445 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:24.833565 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:24.833488 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:24.833639 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:24.833637 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:24.833718 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:24.835582 23581 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1007 22:20:24.841117 23581 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1007 22:20:24.841625 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.841867 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.842045 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.842121 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.842959 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.844370 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.844500 23581 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1007 22:20:24.845090 23581 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1007 22:20:24.845340 23581 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1007 22:20:24.928622 23581 target.go:112] Terraform output is in /tmp/test701806711/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I1007 22:20:24.932987 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:24.933224 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:24.933287 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:24.933344 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:24.933401 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:24.933495 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:24.933526 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:24.933566 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:24.933596 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:24.963799 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:24.967638 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.967735 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:24.967797 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.009202 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:25.111757 23581 executor.go:111] Tasks: 0 done / 108 total; 45 can run W1007 22:20:25.112292 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:25.112407 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.112674 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:25.112724 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:25.112704 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:25.112848 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:25.113052 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.115255 23581 executor.go:111] Tasks: 45 done / 108 total; 25 can run I1007 22:20:25.124484 23581 executor.go:111] Tasks: 70 done / 108 total; 31 can run I1007 22:20:25.124895 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.125085 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.125267 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.125531 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.126818 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.127624 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.127763 23581 executor.go:111] Tasks: 101 done / 108 total; 4 can run I1007 22:20:25.128259 23581 executor.go:111] Tasks: 105 done / 108 total; 3 can run I1007 22:20:25.128561 23581 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1007 22:20:25.198475 23581 target.go:112] Terraform output is in /tmp/test2382790574/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I1007 22:20:25.202780 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:25.203007 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:25.203060 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:25.203112 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:25.203163 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:25.203251 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:25.203281 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:25.203317 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:25.203346 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:25.235348 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:25.239921 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.240034 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.240097 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.282218 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:25.384848 23581 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1007 22:20:25.385163 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:25.385198 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.385203 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:25.385302 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:25.385400 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:25.385826 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.385830 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:25.387723 23581 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1007 22:20:25.388214 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:25.393045 23581 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1007 22:20:25.393380 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.393492 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.393664 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.393737 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.394826 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.395724 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.395859 23581 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1007 22:20:25.396378 23581 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1007 22:20:25.396681 23581 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1007 22:20:25.461662 23581 target.go:112] Terraform output is in /tmp/test3339215829/out --- PASS: TestPrivateSharedSubnet (0.26s) === RUN TestPrivateSharedIP I1007 22:20:25.465873 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:25.466114 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:25.466174 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:25.466279 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:25.466336 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:25.466379 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:25.466411 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:25.466451 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:25.466480 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:25.499111 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:25.502741 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.502841 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.502943 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.543588 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:25.646712 23581 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1007 22:20:25.646994 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:25.647213 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:25.647289 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.647493 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:25.647529 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:25.647626 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:25.647857 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:25.649423 23581 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1007 22:20:25.649867 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:25.654459 23581 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1007 22:20:25.654942 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.655122 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.655295 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.656172 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.658632 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.661532 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.661669 23581 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1007 22:20:25.662118 23581 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1007 22:20:25.662422 23581 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1007 22:20:25.737699 23581 target.go:112] Terraform output is in /tmp/test1569289642/out I1007 22:20:25.741806 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:25.742023 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:25.742079 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:25.742129 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:25.742181 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:25.742229 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:25.742259 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:25.742296 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:25.742331 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:25.770683 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:25.774339 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.774430 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.774491 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.815550 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:25.916697 23581 executor.go:111] Tasks: 0 done / 116 total; 45 can run W1007 22:20:25.917348 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:25.917496 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:25.917631 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:25.917671 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1007 22:20:25.917703 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:25.917666 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:25.917754 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:25.917781 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:25.917692 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:25.917846 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:25.920150 23581 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1007 22:20:25.920629 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:25.928428 23581 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1007 22:20:25.928720 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.929108 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.929292 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.929462 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.929841 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.931445 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:25.931582 23581 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1007 22:20:25.931983 23581 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1007 22:20:25.932263 23581 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1007 22:20:25.937447 23581 target.go:168] Cloudformation output is in /tmp/test3908386588/out --- PASS: TestPrivateSharedIP (0.48s) === RUN TestPrivateDns1 I1007 22:20:25.950739 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:25.950957 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:25.951010 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:25.951060 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:25.951110 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:25.951190 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:25.951219 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:25.951255 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:25.951283 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:25.987462 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:25.987568 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:25.987635 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.146175 23581 executor.go:111] Tasks: 0 done / 120 total; 46 can run W1007 22:20:26.146618 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:26.146754 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:26.147006 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:26.147417 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:26.153951 23581 executor.go:111] Tasks: 46 done / 120 total; 24 can run I1007 22:20:26.154266 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I1007 22:20:26.154307 23581 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I1007 22:20:26.154436 23581 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I1007 22:20:26.154464 23581 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I1007 22:20:26.154611 23581 executor.go:111] Tasks: 70 done / 120 total; 35 can run I1007 22:20:26.155040 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.155304 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.158250 23581 executor.go:111] Tasks: 105 done / 120 total; 5 can run W1007 22:20:26.158319 23581 api.go:53] PageSize not implemented W1007 22:20:26.158388 23581 api.go:53] PageSize not implemented I1007 22:20:26.164367 23581 executor.go:111] Tasks: 110 done / 120 total; 6 can run I1007 22:20:26.164749 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.164994 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.165961 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.168215 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.168367 23581 executor.go:111] Tasks: 116 done / 120 total; 2 can run I1007 22:20:26.168772 23581 executor.go:111] Tasks: 118 done / 120 total; 2 can run I1007 22:20:26.168969 23581 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1007 22:20:26.256081 23581 target.go:112] Terraform output is in /tmp/test4000318077/out --- PASS: TestPrivateDns1 (0.31s) === RUN TestPrivateDns2 I1007 22:20:26.260454 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:26.260720 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:26.260786 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:26.260840 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:26.260896 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:26.260995 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:26.261026 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:26.261068 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:26.261098 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:26.293283 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.293381 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.293447 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.434202 23581 executor.go:111] Tasks: 0 done / 116 total; 44 can run W1007 22:20:26.434653 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:26.435147 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:26.435282 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:26.435336 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:26.437564 23581 executor.go:111] Tasks: 44 done / 116 total; 22 can run I1007 22:20:26.437798 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I1007 22:20:26.437853 23581 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I1007 22:20:26.437987 23581 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I1007 22:20:26.438021 23581 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I1007 22:20:26.438145 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:26.438294 23581 executor.go:111] Tasks: 66 done / 116 total; 35 can run I1007 22:20:26.438565 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.438754 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.445778 23581 executor.go:111] Tasks: 101 done / 116 total; 5 can run W1007 22:20:26.445856 23581 api.go:53] PageSize not implemented W1007 22:20:26.445897 23581 api.go:53] PageSize not implemented I1007 22:20:26.451541 23581 executor.go:111] Tasks: 106 done / 116 total; 6 can run I1007 22:20:26.451820 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.452025 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.453213 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.454334 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.454452 23581 executor.go:111] Tasks: 112 done / 116 total; 2 can run I1007 22:20:26.454771 23581 executor.go:111] Tasks: 114 done / 116 total; 2 can run I1007 22:20:26.454943 23581 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1007 22:20:26.526538 23581 target.go:112] Terraform output is in /tmp/test421306213/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I1007 22:20:26.530834 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:26.531066 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:26.531129 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:26.531232 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:26.531290 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:26.531334 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:26.531364 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:26.531408 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:26.531438 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:26.557042 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:26.560764 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.560859 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.601915 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:26.816115 23581 executor.go:111] Tasks: 0 done / 99 total; 51 can run W1007 22:20:26.816695 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:26.816966 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:26.817249 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:26.817281 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:26.817310 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:26.817429 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:26.819508 23581 executor.go:111] Tasks: 51 done / 99 total; 23 can run I1007 22:20:26.829939 23581 executor.go:111] Tasks: 74 done / 99 total; 21 can run I1007 22:20:26.830309 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.830578 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.831661 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.833208 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.833324 23581 executor.go:111] Tasks: 95 done / 99 total; 2 can run I1007 22:20:26.833610 23581 executor.go:111] Tasks: 97 done / 99 total; 2 can run I1007 22:20:26.833763 23581 executor.go:111] Tasks: 99 done / 99 total; 0 can run I1007 22:20:26.893583 23581 target.go:112] Terraform output is in /tmp/test131687430/out --- PASS: TestDiscoveryFeatureGate (0.37s) === RUN TestVFSServiceAccountIssuerDiscovery I1007 22:20:26.899085 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:26.899341 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:26.899399 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:26.899448 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:26.899502 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:26.899590 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:26.899621 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:26.899657 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:26.899687 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:26.924070 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:26.927397 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.927495 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:26.969405 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:27.068496 23581 executor.go:111] Tasks: 0 done / 90 total; 45 can run W1007 22:20:27.068762 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:27.069082 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:27.069114 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:27.069231 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:27.069321 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:27.069510 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:27.071867 23581 executor.go:111] Tasks: 45 done / 90 total; 20 can run I1007 22:20:27.077376 23581 executor.go:111] Tasks: 65 done / 90 total; 21 can run I1007 22:20:27.077740 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.077939 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.078896 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.080218 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.080342 23581 executor.go:111] Tasks: 86 done / 90 total; 2 can run I1007 22:20:27.080671 23581 executor.go:111] Tasks: 88 done / 90 total; 2 can run I1007 22:20:27.080845 23581 executor.go:111] Tasks: 90 done / 90 total; 0 can run I1007 22:20:27.135413 23581 target.go:112] Terraform output is in /tmp/test3457020752/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.24s) === RUN TestAWSLBController I1007 22:20:27.139568 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:27.139806 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:27.139863 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:27.139917 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:27.139974 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:27.140063 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:27.140094 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:27.140133 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:27.140163 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:27.169471 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:27.172963 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.173083 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.214457 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:27.818689 23581 executor.go:111] Tasks: 0 done / 96 total; 49 can run W1007 22:20:27.819063 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:27.819248 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:27.819391 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:27.819399 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:27.819434 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:27.819583 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:27.826661 23581 executor.go:111] Tasks: 49 done / 96 total; 22 can run I1007 22:20:27.832241 23581 executor.go:111] Tasks: 71 done / 96 total; 21 can run I1007 22:20:27.832558 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.832879 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.834104 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.835326 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.835446 23581 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1007 22:20:27.835773 23581 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1007 22:20:27.835974 23581 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1007 22:20:27.896903 23581 target.go:112] Terraform output is in /tmp/test99851386/out --- PASS: TestAWSLBController (0.76s) === RUN TestManyAddons I1007 22:20:27.903410 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:27.903657 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:27.903720 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:27.903778 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:27.903855 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:27.903950 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:27.903980 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:27.904022 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:27.904052 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:27.930648 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:27.934688 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.934781 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:27.977288 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:28.742044 23581 executor.go:111] Tasks: 0 done / 95 total; 50 can run W1007 22:20:28.742565 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:28.742727 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:28.742762 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:28.742857 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:28.742806 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:28.743113 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:28.747385 23581 executor.go:111] Tasks: 50 done / 95 total; 19 can run I1007 22:20:28.755212 23581 executor.go:111] Tasks: 69 done / 95 total; 22 can run I1007 22:20:28.755544 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.755771 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.756637 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.758296 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.758414 23581 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1007 22:20:28.758717 23581 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1007 22:20:28.758910 23581 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1007 22:20:28.810851 23581 target.go:112] Terraform output is in /tmp/test1800988907/out --- PASS: TestManyAddons (0.92s) === RUN TestManyAddonsCCMIRSA I1007 22:20:28.825188 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:28.825451 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:28.825517 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:28.825631 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:28.825686 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:28.825731 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:28.825761 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:28.825800 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:28.825830 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:28.854741 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:28.858852 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.858960 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:28.899705 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:29.744018 23581 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1007 22:20:29.744751 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:29.744785 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:29.744871 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:29.744957 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:29.745164 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:29.745193 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:29.749167 23581 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1007 22:20:29.759948 23581 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1007 22:20:29.760179 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.760344 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.761817 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.763049 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.763181 23581 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1007 22:20:29.763489 23581 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1007 22:20:29.763741 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:29.832409 23581 target.go:112] Terraform output is in /tmp/test1123356623/out --- PASS: TestManyAddonsCCMIRSA (1.02s) === RUN TestManyAddonsCCMIRSA23 I1007 22:20:29.841335 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:29.841577 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:29.841639 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:29.841753 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:29.841811 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:29.841860 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:29.841891 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:29.841932 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:29.841963 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:29.869412 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1007 22:20:29.873203 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.873297 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:29.916880 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:30.759631 23581 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1007 22:20:30.760278 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:30.760808 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:30.760820 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:30.760923 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:30.760893 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:30.761070 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:30.767251 23581 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1007 22:20:30.772819 23581 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1007 22:20:30.773368 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.773669 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.774898 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.776046 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.776166 23581 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1007 22:20:30.776474 23581 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1007 22:20:30.776650 23581 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1007 22:20:30.848752 23581 target.go:112] Terraform output is in /tmp/test1926982079/out --- PASS: TestManyAddonsCCMIRSA23 (1.02s) === RUN TestManyAddonsCCMIRSA24 I1007 22:20:30.858042 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:30.858285 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:30.858347 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:30.858454 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:30.858509 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:30.858553 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:30.858580 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:30.858618 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:30.858651 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:30.886633 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:30.890294 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.890381 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:30.932241 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:31.798546 23581 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1007 22:20:31.798854 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:31.798881 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:31.798983 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:31.799222 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:31.799493 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:31.799852 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:31.806046 23581 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1007 22:20:31.811492 23581 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1007 22:20:31.811811 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.812043 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.813236 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.814328 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.814460 23581 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1007 22:20:31.814771 23581 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1007 22:20:31.814956 23581 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1007 22:20:31.885890 23581 target.go:112] Terraform output is in /tmp/test3655586529/out --- PASS: TestManyAddonsCCMIRSA24 (1.04s) === RUN TestManyAddonsCCMIRSA25 I1007 22:20:31.894703 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:31.894945 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:31.895014 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:31.895123 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:31.895179 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:31.895229 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:31.895259 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:31.895297 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:31.895331 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:31.922861 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:31.926543 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.926633 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:31.969865 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:32.822387 23581 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1007 22:20:32.822969 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:32.823029 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:32.823070 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:32.823142 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:32.823123 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:32.823476 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:32.829011 23581 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1007 22:20:32.835849 23581 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1007 22:20:32.836201 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.836385 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.837400 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.839357 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.839496 23581 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1007 22:20:32.839837 23581 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1007 22:20:32.840059 23581 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1007 22:20:32.910690 23581 target.go:112] Terraform output is in /tmp/test2266179635/out --- PASS: TestManyAddonsCCMIRSA25 (1.03s) === RUN TestManyAddonsCCMIRSA26 I1007 22:20:32.920940 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:32.921225 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:32.921289 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:32.921342 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:32.921397 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:32.921513 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:32.921545 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:32.921588 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:32.921619 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:32.949031 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* This version of kubernetes is not yet supported; upgrading kops is required (you can bypass this check by exporting KOPS_RUN_TOO_NEW_VERSION) ********************************************************************************* I1007 22:20:32.953159 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.953261 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:32.996075 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:33.856074 23581 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1007 22:20:33.858529 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:33.858799 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:33.862329 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:33.862548 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:33.862585 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:33.862682 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:33.865423 23581 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1007 22:20:33.871290 23581 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1007 22:20:33.871506 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:33.871774 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:33.874576 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:33.878404 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:33.878555 23581 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1007 22:20:33.878989 23581 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1007 22:20:33.879167 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:33.945964 23581 target.go:112] Terraform output is in /tmp/test2566274746/out --- PASS: TestManyAddonsCCMIRSA26 (1.03s) === RUN TestCCM I1007 22:20:33.955068 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:33.955310 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:33.955374 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:33.955431 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:33.955486 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:33.955581 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:33.955613 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:33.955651 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:33.955682 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:33.982325 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:33.985943 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:33.986029 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.028973 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:34.830542 23581 executor.go:111] Tasks: 0 done / 96 total; 51 can run W1007 22:20:34.831031 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:34.831306 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:34.831413 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:34.831346 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:34.831485 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:34.831745 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:34.838605 23581 executor.go:111] Tasks: 51 done / 96 total; 19 can run I1007 22:20:34.844091 23581 executor.go:111] Tasks: 70 done / 96 total; 22 can run I1007 22:20:34.844628 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.844948 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.846002 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.847707 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.847836 23581 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1007 22:20:34.848342 23581 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1007 22:20:34.848667 23581 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1007 22:20:34.909337 23581 target.go:112] Terraform output is in /tmp/test1026552679/out --- PASS: TestCCM (0.96s) === RUN TestExternalDNS I1007 22:20:34.917033 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:34.917294 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:34.917360 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:34.917473 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:34.917527 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:34.917573 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:34.917602 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:34.917641 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:34.917672 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:34.942300 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:34.945630 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.945729 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:34.987123 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:35.087438 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:35.087839 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:35.087872 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:35.087962 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:35.088452 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:35.088489 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:35.088588 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:35.090881 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:35.097983 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:35.098483 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.098709 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.100614 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.103378 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.104205 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:35.105416 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:35.105645 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:35.156921 23581 target.go:112] Terraform output is in /tmp/test4125464844/out I1007 22:20:35.160902 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:35.161121 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:35.161178 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:35.161229 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:35.161283 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:35.161326 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:35.161358 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:35.161395 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:35.161424 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:35.188404 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:35.192085 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.192192 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.232922 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:35.331730 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:35.331959 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:35.331981 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:35.332074 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:35.333617 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:35.335249 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:35.335388 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:35.335405 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:35.335485 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W1007 22:20:35.336445 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:35.339619 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:35.345222 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:35.345534 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.345846 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.346896 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.347886 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.348038 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:35.348454 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:35.348700 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:35.351933 23581 target.go:168] Cloudformation output is in /tmp/test1895632446/out --- PASS: TestExternalDNS (0.44s) === RUN TestExternalDNSIRSA I1007 22:20:35.362049 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:35.362273 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:35.362324 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:35.362370 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:35.362420 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:35.362506 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:35.362537 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:35.362574 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:35.362604 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:35.392523 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:35.395898 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.396008 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.436880 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:35.557261 23581 executor.go:111] Tasks: 0 done / 92 total; 46 can run W1007 22:20:35.557500 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:35.558767 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:35.558998 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:35.560265 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:35.560293 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:35.560364 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:35.565100 23581 executor.go:111] Tasks: 46 done / 92 total; 21 can run I1007 22:20:35.572749 23581 executor.go:111] Tasks: 67 done / 92 total; 21 can run I1007 22:20:35.572971 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.573398 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.574576 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.575364 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.575482 23581 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1007 22:20:35.575785 23581 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1007 22:20:35.575954 23581 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1007 22:20:35.631575 23581 target.go:112] Terraform output is in /tmp/test2301403348/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestKarpenter I1007 22:20:35.636050 23581 featureflag.go:160] FeatureFlag "Karpenter"=true I1007 22:20:35.636099 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:35.636323 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:35.636382 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:35.636436 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:35.636493 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:35.636586 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:35.636616 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:35.636653 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:35.636682 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:35.672610 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:35.676801 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.676902 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.676969 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.717817 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:35.908186 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.908345 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:35.908467 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.026534 23581 executor.go:111] Tasks: 0 done / 108 total; 53 can run W1007 22:20:36.028421 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:36.028535 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.029020 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:36.029105 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:36.029061 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:36.029194 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:36.034330 23581 executor.go:111] Tasks: 53 done / 108 total; 26 can run I1007 22:20:36.040624 23581 executor.go:111] Tasks: 79 done / 108 total; 25 can run I1007 22:20:36.041098 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.041377 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.041686 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.041964 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.044553 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.044903 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.045165 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.049235 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.049446 23581 executor.go:111] Tasks: 104 done / 108 total; 2 can run I1007 22:20:36.050120 23581 executor.go:111] Tasks: 106 done / 108 total; 2 can run I1007 22:20:36.050431 23581 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1007 22:20:36.119674 23581 target.go:112] Terraform output is in /tmp/test3215969156/out I1007 22:20:36.126070 23581 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.49s) === RUN TestSharedSubnet I1007 22:20:36.126208 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:36.126446 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:36.126504 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:36.126559 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:36.126616 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:36.126708 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:36.126737 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:36.126775 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:36.126805 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:36.154916 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:36.158309 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.158411 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.200223 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:36.303736 23581 executor.go:111] Tasks: 0 done / 80 total; 42 can run W1007 22:20:36.304531 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.304750 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:36.304854 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.305088 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:36.305131 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:36.305229 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:36.306574 23581 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1007 22:20:36.307075 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:36.316237 23581 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1007 22:20:36.316604 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.316897 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.318020 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.318908 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.319029 23581 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1007 22:20:36.319322 23581 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1007 22:20:36.319522 23581 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1007 22:20:36.366903 23581 target.go:112] Terraform output is in /tmp/test2313861909/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I1007 22:20:36.370860 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:36.371107 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:36.371167 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:36.371221 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:36.371275 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:36.371365 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:36.371395 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:36.371435 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:36.371464 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:36.399790 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:36.403052 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.403156 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.444198 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:36.549938 23581 executor.go:111] Tasks: 0 done / 84 total; 42 can run W1007 22:20:36.550559 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:36.550930 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.550996 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:36.551100 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:36.551147 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:36.551267 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:36.552636 23581 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1007 22:20:36.552926 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:36.557725 23581 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1007 22:20:36.557996 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.558318 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.559479 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.560453 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.560604 23581 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1007 22:20:36.560927 23581 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1007 22:20:36.561199 23581 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1007 22:20:36.610233 23581 target.go:112] Terraform output is in /tmp/test3748011767/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I1007 22:20:36.614926 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:36.615166 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:36.615224 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:36.615280 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:36.615333 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:36.615429 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:36.615461 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:36.615500 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:36.615529 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:36.648063 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:36.651717 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.651809 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.651868 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.651925 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.692513 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:36.799271 23581 executor.go:111] Tasks: 0 done / 101 total; 51 can run W1007 22:20:36.800025 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:36.800044 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:36.800077 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.800052 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:36.800214 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:36.800133 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:36.800285 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:36.800472 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:36.802885 23581 executor.go:111] Tasks: 51 done / 101 total; 15 can run I1007 22:20:36.813520 23581 executor.go:111] Tasks: 66 done / 101 total; 27 can run I1007 22:20:36.813855 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.814040 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.814308 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.814597 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.815580 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.816675 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.816916 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.817174 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:36.817301 23581 executor.go:111] Tasks: 93 done / 101 total; 4 can run I1007 22:20:36.817777 23581 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1007 22:20:36.818153 23581 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1007 22:20:36.894665 23581 target.go:112] Terraform output is in /tmp/test2533107288/out --- PASS: TestExistingIAM (0.29s) === RUN TestPhaseNetwork I1007 22:20:36.903767 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:36.904054 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:36.904112 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:36.904164 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:36.904224 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:36.904268 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:36.904296 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:36.904333 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:36.904363 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:36.932768 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:36.936477 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.936576 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.936644 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:36.976686 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:37.081921 23581 executor.go:111] Tasks: 0 done / 119 total; 46 can run I1007 22:20:37.082300 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:37.082457 23581 executor.go:111] Tasks: 46 done / 119 total; 28 can run I1007 22:20:37.083060 23581 executor.go:111] Tasks: 74 done / 119 total; 36 can run I1007 22:20:37.083532 23581 executor.go:111] Tasks: 110 done / 119 total; 5 can run I1007 22:20:37.083790 23581 executor.go:111] Tasks: 115 done / 119 total; 4 can run I1007 22:20:37.084035 23581 executor.go:111] Tasks: 119 done / 119 total; 0 can run I1007 22:20:37.092999 23581 target.go:112] Terraform output is in /tmp/test4181122418/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I1007 22:20:37.093787 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:37.094050 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:37.094108 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:37.094214 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:37.094278 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:37.094320 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:37.094350 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:37.094389 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:37.094418 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:37.119763 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:37.123065 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.123164 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.164543 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:37.267692 23581 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1007 22:20:37.268040 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:37.268077 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:37.268193 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:37.269971 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:37.270609 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:37.271165 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:37.275741 23581 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1007 22:20:37.281164 23581 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1007 22:20:37.281484 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.281723 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.282901 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.284386 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.284565 23581 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1007 22:20:37.284909 23581 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1007 22:20:37.285102 23581 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1007 22:20:37.338132 23581 target.go:112] Terraform output is in /tmp/test704027650/out I1007 22:20:37.342190 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:37.342407 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:37.342466 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:37.342517 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:37.342573 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:37.342618 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:37.342648 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:37.342685 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:37.342717 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:37.368575 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:37.371992 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.372099 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.412820 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:37.511060 23581 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1007 22:20:37.511355 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:37.512077 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:37.512199 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:37.512193 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:37.512325 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:37.512238 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:37.512386 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:37.512349 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:37.512450 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I1007 22:20:37.513804 23581 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1007 22:20:37.518801 23581 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1007 22:20:37.519168 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.519460 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.520605 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.524374 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.524548 23581 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1007 22:20:37.525703 23581 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1007 22:20:37.525929 23581 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1007 22:20:37.529561 23581 target.go:168] Cloudformation output is in /tmp/test2864182278/out --- PASS: TestExternalLoadBalancer (0.45s) === RUN TestPhaseIAM integration_test.go:853: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:861: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I1007 22:20:37.540571 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:37.540805 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:37.540859 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:37.540913 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:37.540967 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:37.541113 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:37.541197 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:37.541299 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:37.541371 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:37.574975 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:37.578343 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578425 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578477 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578537 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578585 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578625 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.578671 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.619686 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:37.734968 23581 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1007 22:20:37.735468 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:37.735640 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:37.735660 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:37.735876 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:37.735912 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:37.735952 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:37.736100 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:37.736665 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:37.738267 23581 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1007 22:20:37.748663 23581 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1007 22:20:37.748929 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.749104 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.749419 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.749619 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.750556 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.751300 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.751646 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.752448 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.752582 23581 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1007 22:20:37.753060 23581 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1007 22:20:37.753354 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:37.836138 23581 target.go:112] Terraform output is in /tmp/test1244219188/out I1007 22:20:37.841786 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:37.842037 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:37.842106 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:37.842167 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:37.842235 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:37.842290 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:37.842329 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:37.842381 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:37.842414 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:37.875561 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:37.882153 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882296 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882366 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882428 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882482 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882776 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.882838 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:37.924318 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:38.037323 23581 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1007 22:20:38.037716 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:38.037765 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:38.037884 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:38.038260 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:38.038447 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:38.038497 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:38.038470 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:38.038542 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:38.038649 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:38.038681 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:38.038806 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I1007 22:20:38.040349 23581 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1007 22:20:38.045739 23581 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1007 22:20:38.045952 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.046134 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.046459 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.046753 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.048544 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.051425 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.053370 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.054040 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.054218 23581 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1007 22:20:38.054619 23581 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1007 22:20:38.054857 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:38.060409 23581 target.go:168] Cloudformation output is in /tmp/test2142571076/out --- PASS: TestMixedInstancesASG (0.54s) === RUN TestMixedInstancesSpotASG I1007 22:20:38.077194 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:38.077420 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:38.077471 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:38.077518 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:38.077566 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:38.077646 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:38.077676 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:38.077714 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:38.077740 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:38.107829 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:38.110823 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.110891 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.110935 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.110977 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.111021 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.111057 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.111096 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.152451 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:38.260967 23581 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1007 22:20:38.261704 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:38.261808 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:38.261935 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1007 22:20:38.261978 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:38.261992 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:38.262073 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:38.261985 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:38.262121 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:38.269332 23581 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1007 22:20:38.274799 23581 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1007 22:20:38.275050 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.275334 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.275598 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.275831 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.277832 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.280476 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.282949 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.283071 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.283252 23581 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1007 22:20:38.283797 23581 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1007 22:20:38.284211 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:38.361789 23581 target.go:112] Terraform output is in /tmp/test796833522/out I1007 22:20:38.366922 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:38.367138 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:38.367215 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:38.367268 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:38.367321 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:38.367363 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:38.367391 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:38.367433 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:38.367460 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:38.403912 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:38.408055 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408164 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408230 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408298 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408355 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408439 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.408525 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.450185 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:38.557291 23581 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1007 22:20:38.557758 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:38.557810 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:38.557870 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:38.557930 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:38.557947 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:38.558050 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:38.558024 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I1007 22:20:38.558141 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:38.558181 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:38.558162 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:38.558229 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:38.560113 23581 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1007 22:20:38.565854 23581 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1007 22:20:38.566075 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.566277 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.566535 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.566766 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.567652 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.568504 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.568729 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.570855 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:38.571186 23581 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1007 22:20:38.571965 23581 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1007 22:20:38.572766 23581 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1007 22:20:38.578530 23581 target.go:168] Cloudformation output is in /tmp/test2422959748/out --- PASS: TestMixedInstancesSpotASG (0.52s) === RUN TestAdditionalObjects I1007 22:20:38.595693 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:38.595923 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:38.595978 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:38.596032 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:38.596085 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:38.596166 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:38.596195 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:38.596233 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:38.596261 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:38.622204 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1007 22:20:38.625327 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.625413 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.666134 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:38.830424 23581 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1007 22:20:38.831047 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:38.831051 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:38.831088 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:38.831179 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:38.831260 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:38.831279 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:38.833971 23581 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1007 22:20:38.839484 23581 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1007 22:20:38.840144 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.840431 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.841720 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.845093 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1007 22:20:38.845430 23581 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1007 22:20:38.846474 23581 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1007 22:20:38.846755 23581 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1007 22:20:38.899705 23581 target.go:112] Terraform output is in /tmp/test2835115009/out --- PASS: TestAdditionalObjects (0.31s) === RUN TestContainerd I1007 22:20:38.904096 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:38.904351 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:38.904406 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:38.904514 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:38.904567 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:38.904610 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:38.904640 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:38.904677 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:38.904707 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:38.932455 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:38.935769 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:38.935877 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:38.977999 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:39.078624 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:39.079615 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:39.079599 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:39.079657 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:39.079619 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:39.079779 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:39.079701 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:39.079828 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:39.079810 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:39.079913 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1007 22:20:39.081865 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:39.087240 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:39.087670 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.087897 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.088984 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.089946 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.090088 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:39.090339 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:39.090552 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:39.093405 23581 target.go:168] Cloudformation output is in /tmp/test1082969931/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I1007 22:20:39.102969 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:39.103189 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:39.103242 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:39.103333 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:39.103388 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:39.103433 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:39.103462 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:39.103503 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:39.103533 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:39.132877 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:39.136844 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.136952 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.179760 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:39.277178 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:39.278839 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:39.282687 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:39.282840 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:39.282984 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:39.283004 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:39.283100 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1007 22:20:39.283265 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:39.283305 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:39.283441 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1007 22:20:39.285084 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:39.290720 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:39.291100 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.291373 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.292821 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.294286 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.294452 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:39.294909 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:39.295194 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:39.298380 23581 target.go:168] Cloudformation output is in /tmp/test1122274732/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I1007 22:20:39.309114 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:39.309504 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:39.309620 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:39.309786 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:39.309909 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:39.309995 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:39.310076 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:39.310157 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:39.310208 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:39.338737 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:39.342267 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.342362 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.382562 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:39.487446 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1007 22:20:39.487927 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:39.488045 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:39.488087 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:39.488192 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:39.488385 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:39.489838 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:39.490041 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:39.490070 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:39.490222 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" },{ Key: "Name", Value: "docker.example.com" }] }] } I1007 22:20:39.493721 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:39.499392 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:39.499845 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.500080 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.501464 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.502505 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:39.502651 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:39.502956 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:39.503206 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:39.506454 23581 target.go:168] Cloudformation output is in /tmp/test3260587900/out --- PASS: TestDockerCustom (0.21s) === RUN TestAPIServerNodes I1007 22:20:39.517002 23581 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1007 22:20:39.517048 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:39.517264 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:39.517319 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:39.517413 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:39.517468 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:39.517512 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:39.517545 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:39.517585 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:39.517612 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:39.547849 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1007 22:20:39.551573 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:39.551673 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:39.551739 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.764623 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:40.926420 23581 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1007 22:20:40.926718 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:40.927004 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:40.927270 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:40.927337 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1007 22:20:40.927457 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:40.927369 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:40.927577 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:40.927589 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:40.927491 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:40.927704 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:40.931372 23581 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1007 22:20:40.939712 23581 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1007 22:20:40.940096 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.940236 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.940442 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.941216 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.941435 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.943138 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.943331 23581 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1007 22:20:40.943654 23581 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1007 22:20:40.943943 23581 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1007 22:20:40.947848 23581 target.go:168] Cloudformation output is in /tmp/test3088029489/out I1007 22:20:40.960968 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:40.961177 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:40.961235 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:40.961281 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:40.961337 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:40.961382 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:40.961410 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:40.961451 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:40.961481 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:40.991379 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1007 22:20:40.995216 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.995340 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:40.995441 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.038818 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:41.195076 23581 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1007 22:20:41.195335 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:41.195466 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:41.195687 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:41.195926 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:41.199151 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:41.199276 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:41.199466 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1007 22:20:41.202530 23581 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1007 22:20:41.207897 23581 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1007 22:20:41.208307 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.208571 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.208894 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.209999 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.210104 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.211099 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.211292 23581 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1007 22:20:41.212000 23581 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1007 22:20:41.212224 23581 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1007 22:20:41.279427 23581 target.go:112] Terraform output is in /tmp/test941100538/out I1007 22:20:41.284261 23581 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.77s) === RUN TestNTHQueueProcessor I1007 22:20:41.284395 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:41.284630 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:41.284691 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:41.284799 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:41.284860 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:41.284908 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:41.284938 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:41.284976 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:41.285009 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:41.309881 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:41.313375 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:41.313474 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.528753 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:42.658182 23581 executor.go:111] Tasks: 0 done / 101 total; 45 can run W1007 22:20:42.658452 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:42.658701 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:42.658731 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:42.658820 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:42.658918 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:42.659410 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:42.661661 23581 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1007 22:20:42.666845 23581 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1007 22:20:42.667198 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.667573 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.668281 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.671021 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.671257 23581 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1007 22:20:42.672036 23581 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1007 22:20:42.672767 23581 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1007 22:20:42.733720 23581 target.go:112] Terraform output is in /tmp/test964323298/out I1007 22:20:42.738965 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:42.739192 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:42.739253 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:42.739308 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:42.739364 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:42.739408 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:42.739439 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:42.739476 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:42.739508 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:42.768355 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:42.771678 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.771782 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.815450 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1007 22:20:42.944324 23581 executor.go:111] Tasks: 0 done / 101 total; 45 can run W1007 22:20:42.944595 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1007 22:20:42.945310 23581 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1007 22:20:42.945429 23581 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1007 22:20:42.945408 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:42.945487 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:42.945623 23581 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1007 22:20:42.945673 23581 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1007 22:20:42.945705 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:42.945820 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:42.947763 23581 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1007 22:20:42.956849 23581 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1007 22:20:42.957308 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.957469 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.958622 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.959983 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:42.960122 23581 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1007 22:20:42.960374 23581 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1007 22:20:42.960731 23581 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1007 22:20:42.965106 23581 target.go:168] Cloudformation output is in /tmp/test832477679/out --- PASS: TestNTHQueueProcessor (1.69s) === RUN TestCustomIRSA I1007 22:20:42.975892 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:42.976129 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:42.976180 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:42.976228 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:42.976282 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:42.976368 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:42.976399 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:42.976438 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:42.976468 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:43.005104 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:43.011876 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.011989 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.053199 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:43.593274 23581 executor.go:111] Tasks: 0 done / 100 total; 50 can run I1007 22:20:43.594129 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:43.594194 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:43.594345 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:43.594356 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:43.594358 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:43.594509 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:43.601162 23581 executor.go:111] Tasks: 50 done / 100 total; 25 can run I1007 22:20:43.606444 23581 executor.go:111] Tasks: 75 done / 100 total; 21 can run I1007 22:20:43.606683 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.606876 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.607880 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.609280 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.609497 23581 executor.go:111] Tasks: 96 done / 100 total; 2 can run I1007 22:20:43.610012 23581 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1007 22:20:43.610227 23581 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1007 22:20:43.671225 23581 target.go:112] Terraform output is in /tmp/test3867462115/out --- PASS: TestCustomIRSA (0.70s) === RUN TestClusterNameDigit I1007 22:20:43.678346 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:43.678579 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:43.678633 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:43.678688 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:43.678742 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:43.678828 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:43.678856 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:43.678892 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:43.678921 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:43.707091 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:43.710654 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.710764 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.752082 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:43.856553 23581 executor.go:111] Tasks: 0 done / 95 total; 47 can run I1007 22:20:43.857107 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:43.857385 23581 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1007 22:20:43.857436 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:43.857578 23581 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1007 22:20:43.857658 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target W1007 22:20:43.857678 23581 warmpool.go:124] ASG warm pool is not supported by the terraform target I1007 22:20:43.860061 23581 executor.go:111] Tasks: 47 done / 95 total; 23 can run I1007 22:20:43.869037 23581 executor.go:111] Tasks: 70 done / 95 total; 21 can run I1007 22:20:43.869403 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.869647 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.870780 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.871709 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.871839 23581 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1007 22:20:43.872157 23581 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1007 22:20:43.872423 23581 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1007 22:20:43.930111 23581 target.go:112] Terraform output is in /tmp/test1186876835/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I1007 22:20:43.934778 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:43.935019 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:43.935085 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:43.935139 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:43.935197 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:43.935246 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:43.935276 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:43.935318 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:43.935349 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:43.935421 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I1007 22:20:43.959211 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:43.962446 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:43.962545 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:44.003479 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:44.100314 23581 iamrole.go:89] ListRoles: { } I1007 22:20:44.101938 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:44.102384 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I1007 22:20:44.102493 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" }] } I1007 22:20:44.102530 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.102700 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:44.102853 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I1007 22:20:44.102820 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:44.102968 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1007 22:20:44.102843 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:44.103187 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:44.103259 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:44.103472 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.103808 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.104092 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1007 22:20:44.104137 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.104422 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1007 22:20:44.104563 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:44.104621 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.104784 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:44.104926 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:44.104986 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:44.105049 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:44.105077 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:44.105120 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.105387 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:44.105595 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1007 22:20:44.105570 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.105744 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1007 22:20:44.105847 23581 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1007 22:20:44.105750 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1007 22:20:44.105975 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:44.105979 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1007 22:20:44.106159 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.106170 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:44.106269 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:44.106320 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I1007 22:20:44.106394 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:44.106445 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.106512 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1007 22:20:44.106608 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1007 22:20:44.106858 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:44.106977 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.107085 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.107205 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:44.107286 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:44.107363 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:44.107423 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:44.107462 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.107545 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:44.107591 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:44.107631 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:44.107436 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:44.107700 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:44.107780 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:44.107716 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1007 22:20:44.110631 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:44.110839 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.110963 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.111172 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:44.111235 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:44.111676 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" }] }] } I1007 22:20:44.111977 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.112169 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.112273 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1007 22:20:44.112445 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:44.112532 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.112706 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.112894 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.113015 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:44.113183 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I1007 22:20:44.113343 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.113546 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.113712 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I1007 22:20:44.113886 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:44.113936 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.114067 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.114205 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.114300 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1007 22:20:44.114440 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I1007 22:20:44.114581 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.114627 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.116928 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:44.117291 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:44.117469 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:44.117534 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:44.117600 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:44.118443 23581 dns.go:238] Pre-creating DNS records I1007 22:20:44.118507 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:44.127445 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:44.130961 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.131059 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1007 22:20:44.171869 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:44.271730 23581 iamrole.go:89] ListRoles: { } I1007 22:20:44.273545 23581 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1007 22:20:44.275031 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.276516 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:44.275281 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:44.276683 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:44.276866 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.277131 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.277365 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:44.277642 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:44.277764 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.278142 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.281715 23581 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1007 22:20:44.281860 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.281987 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1007 22:20:44.282135 23581 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1007 22:20:44.282151 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:44.282172 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1007 22:20:44.282208 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:44.282299 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.282437 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1007 22:20:44.282610 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:44.282678 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.282811 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.282951 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:44.287692 23581 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1007 22:20:44.287915 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.288162 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.288324 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.288461 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.288751 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.288920 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.289245 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:44.291760 23581 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1007 22:20:44.292020 23581 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1007 22:20:44.292196 23581 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1007 22:20:44.295025 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:44.295092 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:44.295128 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:44.295160 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:44.295199 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:44.295242 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:44.295281 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:44.295318 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:44.295349 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:44.295386 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:44.295422 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:44.295460 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:44.295505 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:44.295539 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:44.295573 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:44.295609 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:44.295643 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:44.295678 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:44.295714 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:44.295751 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:44.295784 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:44.295816 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:44.295849 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:44.295886 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:44.295934 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:44.295973 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:44.297329 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:44.297360 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:44.297400 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297451 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1007 22:20:44.297508 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297565 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297612 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297670 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297721 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297764 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1007 22:20:44.297812 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297866 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1007 22:20:44.297910 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W1007 22:20:44.297962 23581 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W1007 22:20:44.298000 23581 api.go:53] PageSize not implemented I1007 22:20:44.298024 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:44.298058 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:44.298078 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:44.298107 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:44.298126 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:44.298183 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:44.298213 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:44.298262 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:44.298306 23581 iamrole.go:89] ListRoles: { } I1007 22:20:44.298376 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:44.298423 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:44.298457 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:44.299026 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1007 22:20:44.299116 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I1007 22:20:44.299118 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:44.299164 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I1007 22:20:44.299214 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I1007 22:20:44.299217 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:44.299163 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I1007 22:20:44.299315 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1007 22:20:44.299522 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:44.299545 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:44.299593 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:44.299661 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I1007 22:20:44.299742 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:44.299843 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:44.300071 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:44.300097 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I1007 22:20:44.300171 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:44.300270 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:44.300366 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:44.300401 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I1007 22:20:44.300513 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I1007 22:20:44.300535 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1007 22:20:44.300577 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I1007 22:20:44.300618 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I1007 22:20:44.300662 23581 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I1007 22:20:44.300675 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1007 22:20:44.300699 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I1007 22:20:44.300778 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:44.300865 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:44.300944 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:44.301242 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.37s) === RUN TestLifecycleMinimalOpenstack I1007 22:20:44.301567 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I1007 22:20:44.311484 23581 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:44.384943 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation W1007 22:20:44.538012 23581 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1007 22:20:44.538066 23581 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1007 22:20:44.554884 23581 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1007 22:20:44.564607 23581 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1007 22:20:44.604115 23581 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1007 22:20:44.607795 23581 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1007 22:20:44.612829 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:44.612955 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:44.613317 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1007 22:20:44.623421 23581 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:44.673862 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation W1007 22:20:44.825135 23581 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1007 22:20:44.825183 23581 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1007 22:20:44.844979 23581 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1007 22:20:44.848673 23581 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1007 22:20:44.862630 23581 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1007 22:20:44.864973 23581 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1007 22:20:44.867802 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:44.868708 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:44.870032 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1007 22:20:44.872048 23581 executor.go:111] Tasks: 96 done / 96 total; 0 can run Port:446ee53b-7257-4386-a759-217ec9938d28 ok Port:f20a3de5-fbb1-4222-9c75-b8df7262bee7 ok Instance:40c88256-d2b9-4e88-804b-186fd6e9941e ok SecurityGroup:35f96885-f469-43b6-8adf-c96c9ae333cd ok SecurityGroup:cca8ceb6-be52-4e27-9af0-b13ce68b197d ok Port:f4a8489e-784d-4f49-820d-0f690a5d11d8 ok Subnet:3b2a725f-176f-4c9e-af5d-9515a1fa0258 ok ServerGroup:a7b46b29-8960-418b-99b6-598c216a8ed1 ok ServerGroup:2ef7acdf-6f2d-4ce8-8c98-e14050d7042e ok Network:0a0edbca-08f0-4c11-949b-e75dca1bf5b8 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:2016a60d-24f8-4a58-8930-c51d849e1ee7 ok Instance:c7541625-b5cb-4c03-9df1-0daa8891c1a0 ok SecurityGroup:daff9468-667d-4e40-8b17-811a592514ab ok --- PASS: TestLifecycleMinimalOpenstack (1.68s) === RUN TestLifecycleMinimalGCE I1007 22:20:45.984564 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I1007 22:20:45.986839 23581 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1007 22:20:46.231327 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:46.231418 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:46.231534 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1007 22:20:46.231570 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1007 22:20:46.231596 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:46.231629 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:46.231712 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:46.233470 23581 executor.go:111] Tasks: 0 done / 64 total; 41 can run W1007 22:20:46.241772 23581 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I1007 22:20:46.241876 23581 executor.go:111] Tasks: 40 done / 64 total; 18 can run I1007 22:20:46.247447 23581 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1007 22:20:46.250324 23581 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1007 22:20:46.250588 23581 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1007 22:20:46.250690 23581 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1007 22:20:46.460372 23581 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1007 22:20:46.460453 23581 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1007 22:20:46.460549 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:46.460594 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1007 22:20:46.460616 23581 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1007 22:20:46.460635 23581 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1007 22:20:46.460703 23581 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1007 22:20:46.463597 23581 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1007 22:20:46.468869 23581 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1007 22:20:46.476139 23581 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1007 22:20:46.480092 23581 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1007 22:20:46.480295 23581 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1007 22:20:46.482194 23581 gce.go:98] Scanning zones: [us-test1-a] I1007 22:20:46.482586 23581 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I1007 22:20:46.482611 23581 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1665181246 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1665181246 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.50s) === RUN TestLifecycleFloatingIPOpenstack I1007 22:20:46.484311 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I1007 22:20:46.493140 23581 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:46.566703 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation W1007 22:20:46.716373 23581 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1007 22:20:46.716417 23581 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1007 22:20:46.732417 23581 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1007 22:20:46.743338 23581 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1007 22:20:46.795357 23581 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1007 22:20:46.798520 23581 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1007 22:20:46.800858 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:46.801098 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:46.801349 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1007 22:20:46.805649 23581 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:46.854963 23581 apply_cluster.go:487] Gossip DNS: skipping DNS validation W1007 22:20:47.002645 23581 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1007 22:20:47.002699 23581 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1007 22:20:47.022091 23581 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1007 22:20:47.030905 23581 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1007 22:20:47.037571 23581 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1007 22:20:47.042530 23581 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1007 22:20:47.047666 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:47.047817 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1007 22:20:47.048438 23581 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1007 22:20:47.049950 23581 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:a36bd44f-441d-41df-9d50-8b493f4a9b63 ok Port:af5c5826-d7d2-42ec-9360-96ede724c5c5 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:251bd380-0ba8-4475-8ec3-3c54537a0565 ok Instance:6f2cc773-6a5c-412d-ae02-c6c75a8cec80 ok Port:40feca07-c6de-408a-8f92-573f9728c7d8 ok SecurityGroup:03bfbc54-f48c-4426-92ce-498a9a26daae ok ServerGroup:ff9903d8-9fe8-4255-928d-67b8261697a2 ok Network:bf947ca9-ee77-4214-bc23-f40988136fce ok SecurityGroup:64f37397-ee9d-4580-9773-b2da91416cf7 ok Instance:f4eac56b-c01d-4911-ae59-5cbcec936e7d ok ServerGroup:8c2c20ab-05a6-4923-b2e7-2b347b9d452e ok SecurityGroup:e54a1da5-576b-40c6-a688-5c22fbd568d5 ok Instance:569fd26b-319f-4f14-8532-768b6639c951 ok Port:f51fcf71-831e-4777-acbb-04dfc35f7d31 ok Subnet:a36bd44f-441d-41df-9d50-8b493f4a9b63 ok Router:f45db8dc-1647-47a1-9e55-3bbe9c116fae ok --- PASS: TestLifecycleFloatingIPOpenstack (1.68s) === RUN TestLifecyclePrivateCalico I1007 22:20:48.160369 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:48.160682 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:48.160737 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:48.160788 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:48.160840 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:48.160884 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:48.160915 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:48.160952 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:48.160979 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:48.161053 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I1007 22:20:48.195402 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:48.199449 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.199556 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.199624 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.241056 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:48.686430 23581 iamrole.go:89] ListRoles: { } I1007 22:20:48.688736 23581 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1007 22:20:48.689089 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I1007 22:20:48.689220 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1007 22:20:48.689248 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.689418 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I1007 22:20:48.689410 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.689483 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:48.689514 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1007 22:20:48.689654 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:48.689760 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:48.689820 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I1007 22:20:48.689800 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I1007 22:20:48.689882 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1007 22:20:48.690005 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.690210 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.690245 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.690458 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:48.690615 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:48.690682 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:48.690911 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:48.691006 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:48.691071 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:48.691107 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:48.691163 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:48.691202 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:48.691380 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.692045 23581 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1007 22:20:48.692245 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:48.692282 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1007 22:20:48.692359 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1007 22:20:48.692470 23581 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1007 22:20:48.692528 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.692599 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1007 22:20:48.692632 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1007 22:20:48.692733 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.692789 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1007 22:20:48.692759 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:48.692863 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:48.692917 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1007 22:20:48.693044 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.693059 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:48.693086 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1007 22:20:48.693199 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:48.693247 23581 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I1007 22:20:48.693238 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:48.693356 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1007 22:20:48.693404 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1007 22:20:48.693511 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1007 22:20:48.693572 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:48.693671 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1007 22:20:48.693788 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.693903 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:48.694011 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:48.694008 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:48.694065 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:48.694209 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1007 22:20:48.694361 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1007 22:20:48.694464 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:48.694587 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:48.694648 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:48.694689 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:48.694787 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:48.694832 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:48.694871 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:48.694967 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:48.695023 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:48.695065 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.695142 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:48.695184 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:48.695259 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:48.695299 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:48.695336 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] }], VpcId: "vpc-1" } I1007 22:20:48.695435 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:48.695486 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:48.695531 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:48.695637 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:48.695684 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:48.695734 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1007 22:20:48.695315 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1007 22:20:48.695817 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:48.695892 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:48.696765 23581 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1007 22:20:48.697103 23581 api.go:53] PageSize not implemented I1007 22:20:48.696960 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W1007 22:20:48.697268 23581 api.go:53] PageSize not implemented I1007 22:20:48.697315 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:48.697445 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1007 22:20:48.698141 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:48.698317 23581 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1007 22:20:48.698406 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:48.698526 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1007 22:20:48.698605 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1007 22:20:48.698571 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.698759 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1007 22:20:48.698850 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:48.698955 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:48.698928 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1007 22:20:48.699069 23581 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1007 22:20:48.698989 23581 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.699109 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:48.699185 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1007 22:20:48.699265 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1007 22:20:48.699381 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1007 22:20:48.699272 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.699781 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.700183 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1007 22:20:48.700419 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.700719 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1007 22:20:48.700996 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:48.701297 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.701804 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.701976 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.702138 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.702304 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:48.702448 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.703464 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.703624 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1007 22:20:48.703732 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.703849 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I1007 22:20:48.703993 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1007 22:20:48.704125 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:48.704191 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:48.704234 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" }] }] } I1007 22:20:48.704375 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.704512 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.704648 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1007 22:20:48.704697 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.704837 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:48.704907 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.705040 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.705182 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I1007 22:20:48.705324 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.705410 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1007 22:20:48.705559 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.705646 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.705690 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1007 22:20:48.705731 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:48.705777 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1007 22:20:48.705817 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.705959 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.708685 23581 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1007 22:20:48.708795 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:48.708835 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:48.708973 23581 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:48.709013 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:48.709185 23581 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1007 22:20:48.709299 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:48.709380 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:48.709458 23581 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1007 22:20:48.709603 23581 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1007 22:20:48.709743 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:48.709949 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1007 22:20:48.710041 23581 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1007 22:20:48.710101 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:48.710147 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:48.710224 23581 dns.go:238] Pre-creating DNS records I1007 22:20:48.710261 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:48.716917 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1007 22:20:48.720959 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.721078 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.721180 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:48.761405 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.208974 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.213566 23581 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1007 22:20:49.214365 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.214465 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:49.214623 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.214772 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:49.214809 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:49.214870 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.214912 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.215080 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.215220 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.215303 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.216556 23581 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1007 22:20:49.216684 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1007 22:20:49.216789 23581 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1007 22:20:49.216812 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.216958 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1007 22:20:49.217004 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.217218 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.217365 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.217500 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1007 22:20:49.217661 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:49.217725 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1007 22:20:49.217939 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.218022 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.218152 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.218273 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1007 22:20:49.218452 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.218531 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1007 22:20:49.218735 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:49.218803 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.218954 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:49.219019 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1007 22:20:49.219208 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:49.219325 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:49.221739 23581 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1007 22:20:49.221901 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.221999 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W1007 22:20:49.222028 23581 api.go:53] PageSize not implemented I1007 22:20:49.222047 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1007 22:20:49.222067 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1007 22:20:49.222180 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1007 22:20:49.222159 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.222222 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } W1007 22:20:49.222325 23581 api.go:53] PageSize not implemented I1007 22:20:49.222281 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.222348 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1007 22:20:49.222392 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.222425 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1007 22:20:49.222458 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I1007 22:20:49.222523 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.222664 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.223031 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.223214 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.223476 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.223590 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1007 22:20:49.223642 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.223727 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.226245 23581 executor.go:111] Tasks: 118 done / 127 total; 5 can run W1007 22:20:49.226413 23581 api.go:53] PageSize not implemented I1007 22:20:49.226411 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.226462 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1007 22:20:49.226428 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.226563 23581 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I1007 22:20:49.226617 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:49.226683 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W1007 22:20:49.226845 23581 api.go:53] PageSize not implemented I1007 22:20:49.226898 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1007 22:20:49.227240 23581 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1007 22:20:49.227358 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.227506 23581 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1007 22:20:49.231701 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:49.231769 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:49.231807 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:49.231844 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:49.231878 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:49.231918 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1007 22:20:49.231953 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:49.231988 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:49.232022 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:49.232059 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:49.232097 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:49.232138 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:49.232180 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:49.232220 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1007 22:20:49.232265 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:49.232305 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:49.232341 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:49.232375 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:49.232413 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:49.232449 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:49.232487 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:49.232524 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:49.232563 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:49.232603 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:49.232641 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:49.232681 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.232722 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:49.232759 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:49.232799 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:49.232833 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:49.232868 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:49.232924 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:49.232990 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:49.233049 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:49.235115 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:49.235151 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:49.235194 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235260 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1007 22:20:49.235342 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235407 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235457 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235508 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235551 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235600 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1007 22:20:49.235663 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235722 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:49.235754 23581 natgateway.go:117] DescribeNatGateways: { } I1007 22:20:49.235794 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1007 22:20:49.235836 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W1007 22:20:49.235891 23581 api.go:53] PageSize not implemented I1007 22:20:49.235909 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1007 22:20:49.235962 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:49.235990 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:49.236009 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:49.236036 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:49.236054 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:49.236106 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.236139 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.236185 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:49.236278 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.236365 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:49.236864 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:49.236939 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1007 22:20:49.237015 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:49.237082 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I1007 22:20:49.237776 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:49.237847 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I1007 22:20:49.237853 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } keypair:key-1 ok load-balancer:api-privatecalico-example-0uch4k ok autoscaling-group:bastion.privatecalico.example.com ok I1007 22:20:49.237932 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1007 22:20:49.237949 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-config:lt-1 ok autoscaling-group:nodes.privatecalico.example.com ok I1007 22:20:49.237991 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1007 22:20:49.238072 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I1007 22:20:49.238111 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I1007 22:20:49.238078 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1007 22:20:49.237923 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:bastion-privatecalico-exa-hocohm ok route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I1007 22:20:49.238154 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1007 22:20:49.238014 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W1007 22:20:49.238247 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.238241 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I1007 22:20:49.238270 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:49.238296 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:49.238358 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:49.238395 23581 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1007 22:20:49.238462 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-2 ok I1007 22:20:49.238531 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:49.238586 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1007 22:20:49.238700 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.238713 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1007 22:20:49.238748 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:49.238795 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:49.238873 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.238886 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1007 22:20:49.238969 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I1007 22:20:49.239024 23581 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I1007 22:20:49.239039 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1007 22:20:49.239033 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:49.239078 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } I1007 22:20:49.239114 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } iam-role:masters.privatecalico.example.com ok subnet:subnet-2 ok I1007 22:20:49.239180 23581 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1007 22:20:49.239191 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I1007 22:20:49.239245 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.239278 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I1007 22:20:49.239345 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1007 22:20:49.239406 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } I1007 22:20:49.239299 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } iam-role:nodes.privatecalico.example.com ok W1007 22:20:49.239442 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.239463 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1007 22:20:49.239441 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I1007 22:20:49.239517 23581 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I1007 22:20:49.239531 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1007 22:20:49.239562 23581 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I1007 22:20:49.239533 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.239635 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1007 22:20:49.239762 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.239781 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:49.239867 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1007 22:20:49.239925 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:49.239991 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:49.240063 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:49.240344 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.08s) === RUN TestLifecyclePrivateKopeio I1007 22:20:49.240655 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:49.240898 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:49.240945 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:49.240985 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:49.241036 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:49.241080 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:49.241107 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:49.241144 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:49.241169 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:49.241234 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I1007 22:20:49.270454 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:49.274161 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.274256 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.274320 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.318385 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.435455 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.437857 23581 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1007 22:20:49.438312 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.438409 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.438447 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1007 22:20:49.438607 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I1007 22:20:49.438644 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.438836 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.438839 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I1007 22:20:49.438960 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:49.438962 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1007 22:20:49.438923 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:49.439015 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.439058 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I1007 22:20:49.439185 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.439187 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1007 22:20:49.439357 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:49.439485 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:49.439531 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.439688 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:49.439807 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:49.439850 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.439998 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.440077 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:49.440108 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:49.440136 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:49.440172 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.440566 23581 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1007 22:20:49.440722 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1007 22:20:49.440842 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1007 22:20:49.440782 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.440922 23581 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1007 22:20:49.440915 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I1007 22:20:49.440978 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1007 22:20:49.440988 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.441038 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:49.441123 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:49.441165 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I1007 22:20:49.441189 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1007 22:20:49.441250 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.441193 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I1007 22:20:49.441339 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.441392 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:49.441433 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.441480 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1007 22:20:49.441608 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1007 22:20:49.441679 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1007 22:20:49.441613 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1007 22:20:49.441762 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1007 22:20:49.441872 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.442072 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I1007 22:20:49.442073 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.442271 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I1007 22:20:49.442312 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.442421 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:49.442547 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:49.442523 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1007 22:20:49.442716 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.442834 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1007 22:20:49.442901 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1007 22:20:49.442948 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1007 22:20:49.443040 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1007 22:20:49.443089 23581 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1007 22:20:49.443182 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.443274 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:49.443322 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.443362 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1007 22:20:49.443056 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1007 22:20:49.443564 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.443763 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:49.443848 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1007 22:20:49.444091 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:49.444167 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:49.444221 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:49.444264 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.444385 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:49.444453 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:49.444513 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.444603 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.444711 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1007 22:20:49.444799 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.444882 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:49.444929 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:49.444972 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1007 22:20:49.445025 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:49.445063 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1007 22:20:49.445116 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1007 22:20:49.445153 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:49.445196 23581 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I1007 22:20:49.445275 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:49.445323 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.445361 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:49.445406 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:49.445448 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.445527 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:49.445683 23581 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1007 22:20:49.445887 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } W1007 22:20:49.446052 23581 api.go:53] PageSize not implemented I1007 22:20:49.446069 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W1007 22:20:49.446169 23581 api.go:53] PageSize not implemented I1007 22:20:49.446246 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:49.446221 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1007 22:20:49.446326 23581 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I1007 22:20:49.446401 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1007 22:20:49.446438 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1007 22:20:49.446397 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I1007 22:20:49.446508 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1007 22:20:49.446564 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.446601 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.446692 23581 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1007 22:20:49.446766 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.446802 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1007 22:20:49.446725 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I1007 22:20:49.446857 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1007 22:20:49.446894 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I1007 22:20:49.446955 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1007 22:20:49.447107 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:49.447176 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1007 22:20:49.447390 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1007 22:20:49.447469 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.447535 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.447611 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.447676 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1007 22:20:49.447755 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1007 22:20:49.447910 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I1007 22:20:49.447952 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1007 22:20:49.448084 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1007 22:20:49.448232 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.448361 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:49.448475 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.448610 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" }] }] } I1007 22:20:49.448749 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.448861 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.448936 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.449138 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:49.449247 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1007 22:20:49.449350 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.449493 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.449631 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1007 22:20:49.449762 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.449877 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.449953 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:49.450039 23581 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1007 22:20:49.450088 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1007 22:20:49.450138 23581 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1007 22:20:49.450180 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" }] }] } I1007 22:20:49.450358 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.450455 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1007 22:20:49.450513 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.450672 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.450822 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:49.450865 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1007 22:20:49.450902 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.451035 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.451163 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.451282 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:49.451390 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1007 22:20:49.451524 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:49.451600 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:49.451648 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.451700 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I1007 22:20:49.451737 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1007 22:20:49.451776 23581 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1007 22:20:49.451828 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1007 22:20:49.451874 23581 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1007 22:20:49.451939 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1007 22:20:49.452032 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1007 22:20:49.452093 23581 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1007 22:20:49.453937 23581 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1007 22:20:49.454029 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.454159 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1007 22:20:49.454240 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.454336 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I1007 22:20:49.454404 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I1007 22:20:49.454414 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.454543 23581 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1007 22:20:49.454719 23581 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1007 22:20:49.454774 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.454815 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.454872 23581 dns.go:238] Pre-creating DNS records I1007 22:20:49.454900 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.461394 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:49.465074 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.465171 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.465241 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.505502 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.617253 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.619700 23581 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1007 22:20:49.619905 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.620098 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:49.620211 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.620364 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:49.620401 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:49.620455 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.620520 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.621642 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.622084 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.622244 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.626447 23581 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1007 22:20:49.626622 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.626807 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1007 22:20:49.627698 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1007 22:20:49.626860 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.627776 23581 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1007 22:20:49.627930 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1007 22:20:49.627842 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.628088 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:49.628161 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.628317 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1007 22:20:49.628456 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1007 22:20:49.628579 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1007 22:20:49.628727 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:49.628779 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.628908 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.629041 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:49.629085 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1007 22:20:49.629227 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:49.629273 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1007 22:20:49.629421 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:49.629459 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.629571 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.629674 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.629760 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.629832 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.629867 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.629935 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:49.632706 23581 executor.go:111] Tasks: 78 done / 126 total; 39 can run W1007 22:20:49.632897 23581 api.go:53] PageSize not implemented I1007 22:20:49.632935 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.632959 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.633041 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W1007 22:20:49.633089 23581 api.go:53] PageSize not implemented I1007 22:20:49.633125 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1007 22:20:49.633130 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.633205 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.633270 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1007 22:20:49.633328 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1007 22:20:49.633344 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1007 22:20:49.633417 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1007 22:20:49.633483 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.633662 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.633991 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.634178 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.634399 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.634575 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.634753 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1007 22:20:49.634883 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.635078 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.635291 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:49.637529 23581 executor.go:111] Tasks: 117 done / 126 total; 6 can run W1007 22:20:49.637658 23581 api.go:53] PageSize not implemented I1007 22:20:49.637693 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.637611 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.637730 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.637854 23581 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" W1007 22:20:49.637892 23581 api.go:53] PageSize not implemented I1007 22:20:49.637849 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1007 22:20:49.637931 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.638029 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.638128 23581 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1007 22:20:49.638271 23581 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1007 22:20:49.642308 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:49.642414 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:49.642488 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:49.642545 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:49.642581 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:49.642621 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:49.642658 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:49.642693 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:49.642727 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:49.642765 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:49.642804 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:49.642844 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:49.642884 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1007 22:20:49.642931 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:49.642967 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:49.643004 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:49.643042 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:49.643075 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:49.643113 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1007 22:20:49.643150 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:49.643189 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:49.643226 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:49.643262 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:49.643300 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:49.643338 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.643377 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:49.643411 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:49.643450 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1007 22:20:49.643487 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1007 22:20:49.643525 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:49.643559 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:49.643595 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:49.643633 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:49.643672 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:49.643710 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:49.645436 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:49.645467 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:49.645509 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645576 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1007 22:20:49.645656 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645718 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645767 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645818 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645859 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.645913 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1007 22:20:49.645981 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.646051 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:49.646084 23581 natgateway.go:117] DescribeNatGateways: { } I1007 22:20:49.646121 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1007 22:20:49.646163 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W1007 22:20:49.646216 23581 api.go:53] PageSize not implemented I1007 22:20:49.646234 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1007 22:20:49.646285 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:49.646312 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:49.646332 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:49.646355 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:49.646373 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:49.646419 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.646446 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.646494 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:49.646533 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.646577 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:49.646624 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:49.646675 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-2","rtb-1"] } I1007 22:20:49.646720 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1007 22:20:49.646753 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } autoscaling-config:lt-1 ok I1007 22:20:49.647418 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } autoscaling-group:bastion.privatekopeio.example.com ok I1007 22:20:49.647490 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I1007 22:20:49.647553 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I1007 22:20:49.647533 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1007 22:20:49.647681 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I1007 22:20:49.647670 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I1007 22:20:49.647533 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I1007 22:20:49.647744 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1007 22:20:49.647643 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W1007 22:20:49.647870 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.647859 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I1007 22:20:49.647893 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:49.647915 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1007 22:20:49.647931 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:49.647966 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I1007 22:20:49.647989 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:49.648019 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:49.648081 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.648135 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:49.648219 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.648234 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-3 ok I1007 22:20:49.648275 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1007 22:20:49.648334 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:49.648380 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1007 22:20:49.648489 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.648502 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1007 22:20:49.648603 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:49.648621 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I1007 22:20:49.648709 23581 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1007 22:20:49.648659 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1007 22:20:49.648762 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.648777 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1007 22:20:49.648727 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1007 22:20:49.648819 23581 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I1007 22:20:49.648811 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I1007 22:20:49.648854 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I1007 22:20:49.648903 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1007 22:20:49.648878 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1007 22:20:49.648917 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1007 22:20:49.648950 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I1007 22:20:49.648930 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.648975 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I1007 22:20:49.649020 23581 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I1007 22:20:49.649034 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1007 22:20:49.648999 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I1007 22:20:49.649063 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok W1007 22:20:49.649080 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:49.649095 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:49.649130 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1007 22:20:49.649190 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:49.649272 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1007 22:20:49.649327 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I1007 22:20:49.649426 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:49.649516 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:49.649592 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:49.649854 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.41s) === RUN TestLifecycleIPv6 I1007 22:20:49.650130 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:49.650390 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:49.650439 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:49.650514 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:49.650568 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:49.650610 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:49.650640 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:49.650679 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:49.650706 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:49.650769 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I1007 22:20:49.675823 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:49.679583 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.679678 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.720315 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.895419 23581 iamrole.go:89] ListRoles: { } I1007 22:20:49.897566 23581 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1007 22:20:49.897980 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.898026 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:49.898134 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.898208 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:49.898317 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.898471 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1007 22:20:49.898561 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1007 22:20:49.898551 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.898667 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.898849 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I1007 22:20:49.898927 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.899053 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:49.899031 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] } I1007 22:20:49.899105 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.899280 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:49.899449 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:49.899515 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:49.899549 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:49.899580 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:49.899620 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.899656 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I1007 22:20:49.899814 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:49.899939 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:49.901305 23581 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1007 22:20:49.901469 23581 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1007 22:20:49.901446 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.901628 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I1007 22:20:49.901663 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1007 22:20:49.901718 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.901774 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1007 22:20:49.901789 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1007 22:20:49.901831 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1007 22:20:49.901976 23581 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1007 22:20:49.901882 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1007 22:20:49.901983 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1007 22:20:49.902156 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1007 22:20:49.902190 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1007 22:20:49.902404 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.902508 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:49.902561 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:49.902612 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.902795 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:49.902929 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:49.902993 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:49.903132 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:49.903206 23581 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I1007 22:20:49.903321 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.903449 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:49.903520 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1007 22:20:49.903802 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.903932 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:49.903981 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:49.904059 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:49.904152 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:49.904216 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:49.904259 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:49.904309 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:49.904377 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] } I1007 22:20:49.904470 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:49.904523 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:49.904661 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1007 22:20:49.906705 23581 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1007 22:20:49.906880 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.906988 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.907092 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I1007 22:20:49.907350 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:49.907430 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.907710 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.907874 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.908010 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1007 22:20:49.908186 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I1007 22:20:49.908332 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:49.908396 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.908548 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.908637 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:49.908678 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.908763 23581 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I1007 22:20:49.908817 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" }] }] } I1007 22:20:49.908956 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:49.909055 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.909135 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.909282 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:49.909379 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.909524 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.909588 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.909772 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.909934 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I1007 22:20:49.910086 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.910233 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.910383 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1007 22:20:49.910444 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1007 22:20:49.910620 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.910761 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" }] }] } I1007 22:20:49.910888 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.910953 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:49.910994 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I1007 22:20:49.911136 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1007 22:20:49.911228 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:49.911287 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.911393 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.912563 23581 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1007 22:20:49.912640 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.912726 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I1007 22:20:49.912789 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:49.912833 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } W1007 22:20:49.912813 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:49.912875 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.913507 23581 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.913869 23581 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1007 22:20:49.913999 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:49.913834 23581 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I1007 22:20:49.914062 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:49.914179 23581 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I1007 22:20:49.914279 23581 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1007 22:20:49.914336 23581 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1007 22:20:49.914550 23581 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1007 22:20:49.914767 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.914806 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:49.915132 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.915290 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1007 22:20:49.915409 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.915672 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:49.915919 23581 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1007 22:20:49.916039 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:49.916130 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.916450 23581 dns.go:238] Pre-creating DNS records I1007 22:20:49.916786 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:49.924670 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:49.928339 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.928436 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:49.969918 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.140509 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.142341 23581 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1007 22:20:50.142563 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:50.142824 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.142949 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:50.142981 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:50.143016 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.143194 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.143234 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.143293 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.143370 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.143588 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.146792 23581 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1007 22:20:50.147202 23581 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1007 22:20:50.147042 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1007 22:20:50.147094 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1007 22:20:50.147343 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:50.147470 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1007 22:20:50.147370 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1007 22:20:50.147973 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.148309 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1007 22:20:50.149326 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:50.149383 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] }] } I1007 22:20:50.150311 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.150430 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.150588 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1007 22:20:50.150762 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.150863 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.154755 23581 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1007 22:20:50.155055 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.155180 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:50.155367 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:50.155458 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1007 22:20:50.155523 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1007 22:20:50.155732 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:50.155910 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.156138 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.156234 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.156346 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.156440 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.156510 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:50.156550 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.156701 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.159300 23581 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1007 22:20:50.159381 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.159436 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:50.159530 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:50.159552 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1007 22:20:50.159594 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1007 22:20:50.159564 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.159633 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1007 22:20:50.159669 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:50.159681 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1007 22:20:50.159717 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1007 22:20:50.159859 23581 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1007 22:20:50.159938 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:50.160020 23581 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1007 22:20:50.160045 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:50.159970 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.160066 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1007 22:20:50.160072 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:50.160201 23581 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1007 22:20:50.160227 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:50.160251 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1007 22:20:50.160349 23581 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1007 22:20:50.163430 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:50.163495 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:50.163535 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:50.163566 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:50.163602 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:50.163646 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1007 22:20:50.163685 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:50.163720 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:50.163754 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:50.163789 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:50.163846 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:50.163902 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:50.163967 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:50.164016 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:50.164066 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:50.164104 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:50.164138 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:50.164183 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:50.164233 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:50.164275 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:50.164312 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:50.164351 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:50.164388 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:50.164433 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:50.164470 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:50.164505 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:50.164539 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:50.164577 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:50.164617 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:50.164654 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:50.166247 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:50.166281 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:50.166326 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166386 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1007 22:20:50.166450 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166509 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166561 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166615 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166658 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166703 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1007 22:20:50.166760 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166810 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:50.166839 23581 natgateway.go:117] DescribeNatGateways: { } I1007 22:20:50.166887 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1007 22:20:50.166937 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W1007 22:20:50.166995 23581 api.go:53] PageSize not implemented I1007 22:20:50.167015 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:50.167044 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:50.167058 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1007 22:20:50.167098 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:50.167122 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:50.167138 23581 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I1007 22:20:50.167227 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.167257 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.167303 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:50.167341 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.167385 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:50.167436 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:50.167479 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1007 22:20:50.167518 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:50.167558 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I1007 22:20:50.168139 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:50.168165 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok I1007 22:20:50.168186 23581 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I1007 22:20:50.168251 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1007 22:20:50.168253 23581 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } iam-instance-profile:nodes.minimal-ipv6.example.com ok nat-gateway:nat-1 ok I1007 22:20:50.168340 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:50.168391 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1007 22:20:50.168458 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I1007 22:20:50.168409 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } I1007 22:20:50.168470 23581 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } route-table:rtb-2 ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I1007 22:20:50.168329 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.168537 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok I1007 22:20:50.168600 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:50.168638 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:50.168704 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:50.168737 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.168791 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 } ] } W1007 22:20:50.169091 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:50.169113 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:50.169147 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I1007 22:20:50.169271 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:50.169356 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1007 22:20:50.169554 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:50.169570 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1007 22:20:50.169662 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I1007 22:20:50.169720 23581 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1007 22:20:50.169687 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:50.169736 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1007 22:20:50.169767 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I1007 22:20:50.169771 23581 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1007 22:20:50.169800 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I1007 22:20:50.169846 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1007 22:20:50.169859 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1007 22:20:50.169888 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I1007 22:20:50.169942 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:50.170020 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:50.170113 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:50.170418 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.52s) === RUN TestLifecycleSharedVPC I1007 22:20:50.170819 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:50.171072 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:50.171117 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:50.171156 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:50.171203 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:50.171246 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:50.171275 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:50.171311 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:50.171336 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:50.171399 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I1007 22:20:50.195310 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:50.198546 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.198636 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.239551 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.340223 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.341759 23581 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1007 22:20:50.342006 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.342222 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:50.342373 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:50.342041 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I1007 22:20:50.342449 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I1007 22:20:50.342471 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:50.342576 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.342633 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I1007 22:20:50.342646 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.342760 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] } I1007 22:20:50.342865 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:50.343018 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:50.343080 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.343253 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:50.343386 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:50.343422 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:50.343476 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.345005 23581 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1007 22:20:50.345147 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1007 22:20:50.345262 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1007 22:20:50.345401 23581 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1007 22:20:50.345365 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:50.345440 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1007 22:20:50.345522 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1007 22:20:50.345486 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.345742 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I1007 22:20:50.345739 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1007 22:20:50.346040 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.346158 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:50.346100 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1007 22:20:50.346282 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I1007 22:20:50.346292 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:50.346438 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:50.346504 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:50.346558 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:50.346605 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:50.346650 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:50.346782 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:50.346840 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.346884 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:50.346929 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I1007 22:20:50.347526 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1007 22:20:50.350352 23581 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1007 22:20:50.350516 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.350692 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:50.350741 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:50.350859 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1007 22:20:50.351126 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I1007 22:20:50.351352 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.351498 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" }] }] } I1007 22:20:50.351641 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.351851 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1007 22:20:50.352001 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:50.352134 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.352262 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.352329 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.352415 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.352519 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.352594 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.352737 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I1007 22:20:50.352872 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1007 22:20:50.352924 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1007 22:20:50.352967 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.353105 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1007 22:20:50.353250 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.353389 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.353592 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1007 22:20:50.353751 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.360260 23581 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1007 22:20:50.360558 23581 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1007 22:20:50.360789 23581 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1007 22:20:50.360843 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.360880 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.360940 23581 dns.go:238] Pre-creating DNS records I1007 22:20:50.360970 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.367625 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:50.371394 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.371519 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.411958 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.508388 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.509905 23581 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1007 22:20:50.510331 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:50.510517 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:50.510583 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:50.510654 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.510761 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:50.510829 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.510976 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.511029 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:50.514868 23581 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1007 22:20:50.515106 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1007 22:20:50.515088 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:50.515317 23581 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1007 22:20:50.515322 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1007 22:20:50.516391 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.516473 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.516645 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:50.516698 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.517660 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1007 22:20:50.517885 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:50.521819 23581 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1007 22:20:50.522089 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.522333 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.522530 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.522642 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.522797 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.522944 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.523116 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.525953 23581 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1007 22:20:50.526182 23581 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1007 22:20:50.526352 23581 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1007 22:20:50.529591 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:50.529656 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:50.529691 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:50.529723 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:50.529758 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:50.529793 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:50.529827 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:50.529859 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:50.529897 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:50.529935 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:50.529977 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:50.530007 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:50.530041 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:50.530075 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:50.530106 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:50.530138 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:50.530172 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:50.530209 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:50.530241 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:50.530272 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:50.530304 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:50.530341 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:50.530377 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:50.531788 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:50.531817 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:50.531854 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.531903 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1007 22:20:50.531956 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532014 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532056 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532097 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532138 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532176 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1007 22:20:50.532222 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532275 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1007 22:20:50.532310 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W1007 22:20:50.532358 23581 api.go:53] PageSize not implemented I1007 22:20:50.532379 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:50.532406 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:50.532425 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:50.532453 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:50.532471 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:50.532540 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.532570 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.532614 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:50.532653 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.532691 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:50.532728 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:50.532759 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.533150 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:50.533236 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:50.533219 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } autoscaling-group:nodes.sharedvpc.example.com ok autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I1007 22:20:50.533312 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I1007 22:20:50.533300 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:50.533360 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1007 22:20:50.533453 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I1007 22:20:50.533431 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:50.533336 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I1007 22:20:50.533536 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:50.533759 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:50.533784 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I1007 22:20:50.533867 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.533923 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:50.534033 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:50.534046 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:50.534116 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I1007 22:20:50.534172 23581 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I1007 22:20:50.534187 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1007 22:20:50.534218 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I1007 22:20:50.534179 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:50.534246 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I1007 22:20:50.534308 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1007 22:20:50.534323 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1007 22:20:50.534353 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I1007 22:20:50.534415 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:50.534692 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.36s) === RUN TestLifecycleComplex I1007 22:20:50.534972 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:50.535210 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:50.535257 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:50.535301 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:50.535350 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:50.535389 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:50.535419 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:50.535456 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:50.535483 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:50.535546 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I1007 22:20:50.570354 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:50.574849 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.574939 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.617218 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.806581 23581 iamrole.go:89] ListRoles: { } I1007 22:20:50.808690 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1007 22:20:50.809211 23581 api.go:53] PageSize not implemented I1007 22:20:50.809259 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1007 22:20:50.809350 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I1007 22:20:50.809341 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:50.809450 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I1007 22:20:50.809536 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:50.809490 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:50.809571 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I1007 22:20:50.809618 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.811695 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.811866 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I1007 22:20:50.811982 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:50.812031 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:50.812136 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:50.812177 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:50.812249 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:50.812315 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.812449 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1007 22:20:50.812459 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I1007 22:20:50.812875 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:50.812985 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:50.813054 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:50.813188 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.813268 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:50.813505 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:50.815496 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:50.815625 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:50.815640 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.815750 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.815817 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1007 22:20:50.815925 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:50.815883 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:50.815938 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.816021 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:50.816000 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1007 22:20:50.816060 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1007 22:20:50.816123 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:50.816193 23581 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1007 22:20:50.816213 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1007 22:20:50.816327 23581 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1007 22:20:50.816402 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:50.816485 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:50.816542 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:50.816597 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1007 22:20:50.816761 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1007 22:20:50.816900 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:50.816957 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:50.817000 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.817197 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.817350 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:50.817265 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1007 22:20:50.817447 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1007 22:20:50.817493 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:50.817515 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:50.817649 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:50.817694 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.817799 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.817894 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:50.817996 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I1007 22:20:50.818137 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:50.817883 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1007 22:20:50.818298 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1007 22:20:50.818427 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:50.818499 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:50.818555 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:50.818607 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1007 22:20:50.818718 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I1007 22:20:50.818821 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:50.818876 23581 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I1007 22:20:50.818931 23581 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I1007 22:20:50.818979 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:50.819028 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1007 22:20:50.819177 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:50.819177 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1007 22:20:50.819256 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:50.819312 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1007 22:20:50.819416 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:50.819465 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:50.819502 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:50.819548 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:50.819601 23581 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1007 22:20:50.821305 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:50.821471 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.821572 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.821587 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:50.821787 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:50.821715 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:50.821894 23581 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-3" }], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api.complex.example.com" } ], Type: "network" } I1007 22:20:50.821936 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.822027 23581 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1007 22:20:50.822126 23581 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1007 22:20:50.822125 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:50.822199 23581 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:50.822324 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1007 22:20:50.822649 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1007 22:20:50.822800 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1007 22:20:50.823045 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1007 22:20:50.823227 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:50.823342 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:50.823474 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1007 22:20:50.823718 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:50.823863 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:50.823988 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I1007 22:20:50.824163 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I1007 22:20:50.824280 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1007 22:20:50.824323 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I1007 22:20:50.824505 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1007 22:20:50.824699 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1007 22:20:50.824898 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1007 22:20:50.825015 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I1007 22:20:50.825320 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1007 22:20:50.825546 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1007 22:20:50.825721 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:50.825764 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I1007 22:20:50.825958 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I1007 22:20:50.826086 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1007 22:20:50.826238 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1007 22:20:50.826327 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" } ] }] } I1007 22:20:50.826480 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.826573 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I1007 22:20:50.826626 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:50.826670 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I1007 22:20:50.826821 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I1007 22:20:50.826910 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1007 22:20:50.827058 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1007 22:20:50.827245 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1007 22:20:50.827414 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I1007 22:20:50.827583 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.827667 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I1007 22:20:50.827714 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1007 22:20:50.827807 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I1007 22:20:50.827895 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1007 22:20:50.827934 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1007 22:20:50.827967 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1007 22:20:50.828091 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.828148 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.831476 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:50.831578 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:50.831803 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:50.831942 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:50.832098 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:50.832159 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:50.832197 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.832249 23581 dns.go:238] Pre-creating DNS records I1007 22:20:50.832276 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:50.840316 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:50.844955 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.845057 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:50.886096 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.079192 23581 iamrole.go:89] ListRoles: { } I1007 22:20:51.081541 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1007 22:20:51.081927 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W1007 22:20:51.082079 23581 api.go:53] PageSize not implemented I1007 22:20:51.082131 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.082340 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:51.082420 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.082647 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.082819 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:51.082866 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:51.082959 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:51.083155 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:51.083263 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:51.083362 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.083551 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.084802 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:51.084966 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.085258 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:51.085141 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:51.085315 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:51.085220 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:51.085439 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.085357 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:51.085527 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:51.086488 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.086596 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.086743 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:51.086954 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:51.087068 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:51.087436 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.087801 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.088209 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:51.088301 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1007 22:20:51.088510 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.088679 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.088906 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.089103 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:51.089307 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.089507 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.089717 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:51.089764 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:51.094604 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:51.094788 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.094887 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:51.095012 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:51.095050 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.095059 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.095128 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.095173 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.095213 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.095250 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.095284 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.095267 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.095460 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.095605 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.095754 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.096152 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.096296 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.096458 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.096532 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.096608 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.098115 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:51.098200 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:51.098289 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:51.098315 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:51.098338 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.098418 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:51.098572 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I1007 22:20:51.105022 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.109497 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.122050 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:51.126477 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.126578 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.167219 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.364900 23581 iamrole.go:89] ListRoles: { } I1007 22:20:51.367299 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1007 22:20:51.367507 23581 api.go:53] PageSize not implemented I1007 22:20:51.367640 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:51.367884 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:51.368114 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.368321 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.368619 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:51.368808 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:51.368979 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.369165 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:51.369223 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:51.369270 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:51.369383 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.369561 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.370210 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:51.370357 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.370524 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:51.370508 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.370593 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:51.370690 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.370782 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:51.370694 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:51.370835 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:51.370939 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.370948 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:51.371052 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.371166 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.371252 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.371339 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.371420 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:51.371606 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:51.371738 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:51.371834 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.372065 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:51.372138 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.372363 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:51.372414 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.372556 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.372687 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:51.372840 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:51.376354 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:51.376671 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.376736 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:51.376820 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:51.376786 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.376845 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.376905 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.376964 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.377025 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.376989 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.377073 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.377117 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.377303 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.377372 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I1007 22:20:51.377506 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.377512 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.377544 23581 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I1007 22:20:51.377596 23581 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I1007 22:20:51.377641 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.377647 23581 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1007 22:20:51.377731 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.377775 23581 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1007 22:20:51.377863 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.377909 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.378081 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.378195 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.378347 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.378416 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.385007 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:51.385159 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:51.385273 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:51.385300 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:51.385334 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.385435 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:51.385597 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1007 22:20:51.385654 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.385694 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:51.385757 23581 dns.go:238] Pre-creating DNS records I1007 22:20:51.393650 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:51.398725 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.398833 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.439179 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.634492 23581 iamrole.go:89] ListRoles: { } I1007 22:20:51.636875 23581 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1007 22:20:51.637193 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.637358 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.637512 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:51.637552 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W1007 22:20:51.637604 23581 api.go:53] PageSize not implemented I1007 22:20:51.637624 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:51.637695 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:51.637895 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.638096 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.638154 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:51.638261 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:51.638360 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.638498 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:51.640125 23581 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1007 22:20:51.640255 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.640409 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.640455 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:51.640580 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:51.640620 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.640687 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:51.640620 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.640735 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:51.640788 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.640848 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.640895 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I1007 22:20:51.641054 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:51.641108 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.641251 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.641456 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:51.641527 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.641679 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.641774 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:51.641957 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:51.642017 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:51.642162 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:51.642226 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.642376 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:51.642435 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:51.642604 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:51.645917 23581 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1007 22:20:51.646177 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.646393 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.646411 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:51.646583 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:51.646544 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.646607 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.646674 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.646716 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.646769 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.646819 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.646859 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.646826 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.647307 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.647418 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.647572 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.647713 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.648133 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.648285 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.648436 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.651166 23581 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1007 22:20:51.651334 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:51.651438 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:51.651469 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:51.651501 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.651599 23581 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1007 22:20:51.651743 23581 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I1007 22:20:51.655512 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.659168 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.671266 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:51.675666 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.675768 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.716670 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.908886 23581 iamrole.go:89] ListRoles: { } I1007 22:20:51.910948 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1007 22:20:51.911131 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:51.911297 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.911429 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:51.911544 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:51.911682 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W1007 22:20:51.913295 23581 api.go:53] PageSize not implemented I1007 22:20:51.911855 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.913784 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:51.913887 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:51.913997 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:51.914192 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:51.914329 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.914536 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.914809 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.918518 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:51.918681 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1007 22:20:51.918811 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:51.918778 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:51.918881 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.918915 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:51.918879 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.918999 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:51.919065 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.919021 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:51.919215 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:51.919292 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:51.919446 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:51.919496 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.919649 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:51.919701 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.919851 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.920025 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:51.920069 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.920166 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.920318 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:51.920370 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.920631 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:51.920786 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:51.921010 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:51.921288 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:51.924468 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:51.924658 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.924758 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.924835 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:51.924950 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:51.924978 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.924921 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.925066 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.925129 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.925191 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:51.925245 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:51.925288 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:51.925230 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.925412 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.925771 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.925951 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.926315 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.926438 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:51.926582 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.926699 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:51.929361 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.929437 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.929669 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:51.929840 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:51.929954 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:51.929997 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:51.930047 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:51.930176 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:51.930336 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:51.930392 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:51.930431 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:51.930506 23581 dns.go:238] Pre-creating DNS records I1007 22:20:51.938797 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:51.943332 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.943436 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:51.984866 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.169393 23581 iamrole.go:89] ListRoles: { } I1007 22:20:52.171696 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1007 22:20:52.171986 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1007 22:20:52.172176 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1007 22:20:52.172286 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:52.172394 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:52.172443 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W1007 22:20:52.172477 23581 api.go:53] PageSize not implemented I1007 22:20:52.172498 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:52.172648 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.172703 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:52.172812 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:52.172910 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.173074 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.173203 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:52.173303 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:52.174399 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:52.174549 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.174723 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:52.174701 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:52.174794 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:52.174890 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.174915 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:52.174858 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.174971 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:52.174992 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.175062 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.175167 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:52.175354 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:52.175418 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.175566 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:52.175715 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:52.175789 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.175887 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.176033 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:52.176073 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.176157 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:52.176204 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.176340 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:52.176392 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.176544 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.176718 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:52.179657 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:52.179867 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.179995 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.180045 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:52.180119 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:52.180137 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.180178 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.180212 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.180174 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.180256 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.180291 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.180323 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.180315 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.180434 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.180566 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.180721 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.180928 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.181260 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.181578 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.181771 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.184586 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:52.184715 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:52.184816 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:52.184846 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:52.184877 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.184971 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:52.185127 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I1007 22:20:52.190181 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.193779 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.193890 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.203466 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:52.208861 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.209003 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.209125 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.250932 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.440911 23581 iamrole.go:89] ListRoles: { } I1007 22:20:52.442641 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1007 22:20:52.442979 23581 api.go:53] PageSize not implemented I1007 22:20:52.443098 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:52.443207 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:52.443188 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:52.443321 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:52.444939 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.445085 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:52.445188 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.445287 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:52.445316 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:52.445703 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.445864 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:52.445956 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.446215 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.450325 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:52.450502 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:52.450600 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.450725 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:52.450726 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:52.450858 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.450903 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:52.450929 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.450960 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:52.451086 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.451160 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:52.451232 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.451387 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.451491 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:52.451669 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:52.451733 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.451918 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.452169 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.452419 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:52.452496 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:52.452720 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.452900 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.453080 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.453261 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:52.453325 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:52.453397 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:52.456998 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:52.457367 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.457558 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:52.457646 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:52.457573 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.457667 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.457720 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.457762 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.457803 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.457781 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.457859 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.457894 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.457953 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.458215 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.458332 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.458511 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.458606 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.458718 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.459057 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.459249 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.462017 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:52.462184 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:52.462292 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:52.462320 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:52.462346 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.462467 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:52.462617 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:52.462666 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.462699 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:52.462752 23581 dns.go:238] Pre-creating DNS records I1007 22:20:52.471137 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:52.475872 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.475980 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.476044 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.516599 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.711303 23581 iamrole.go:89] ListRoles: { } I1007 22:20:52.713593 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1007 22:20:52.713969 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:52.714116 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:52.714287 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:52.714411 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:52.714463 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1007 22:20:52.714638 23581 api.go:53] PageSize not implemented I1007 22:20:52.714587 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:52.714793 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.714995 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.715139 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.715276 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.715385 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:52.715415 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:52.715472 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.716189 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:52.716326 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:52.716329 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I1007 22:20:52.716414 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.716526 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:52.716565 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:52.716585 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.716609 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:52.716697 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.716707 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.716675 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:52.716815 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.716977 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:52.717021 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.717113 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:52.717342 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:52.717410 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.717507 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.717647 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:52.717807 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:52.717848 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.717959 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.718029 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:52.718151 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.718280 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:52.718361 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:52.721912 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:52.722025 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.722204 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.722265 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:52.722354 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:52.722375 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.722429 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.722466 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.722452 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.722508 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.722616 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.722642 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.722697 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.722899 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.723078 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.723175 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.723374 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.723551 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.723611 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.723707 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.726321 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:52.726413 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:52.726505 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:52.726532 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:52.726557 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.726665 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:52.726874 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I1007 22:20:52.729964 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.733526 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.733621 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.733677 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.743666 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:52.748202 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.748297 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.748355 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.748413 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.789929 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.979810 23581 iamrole.go:89] ListRoles: { } I1007 22:20:52.982296 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1007 22:20:52.982502 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:52.982501 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.982704 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1007 22:20:52.982897 23581 api.go:53] PageSize not implemented I1007 22:20:52.982873 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:52.983008 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:52.983116 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:52.983331 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:52.983471 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.983598 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:52.983631 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:52.983676 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:52.983846 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.984017 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.985570 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:52.985713 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.985876 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:52.985853 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:52.985964 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.986028 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:52.986069 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:52.985993 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.986151 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:52.986299 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.986154 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.986815 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1007 22:20:52.987017 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.987209 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.987437 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:52.987500 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.987640 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:52.987868 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:52.987989 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:52.988136 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:52.988202 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.988327 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:52.988463 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:52.988520 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:52.988604 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:52.988684 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:52.991383 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:52.991523 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.991672 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:52.991752 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:52.991687 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.991770 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.991868 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.991938 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.992010 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:52.992070 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:52.992129 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:52.992233 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.992397 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.992561 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.992721 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.992891 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.992965 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:52.993049 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.993117 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:52.993221 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:52.995910 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:52.996016 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:52.996108 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:52.996134 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:52.996160 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:52.996302 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:52.996436 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:52.996485 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:52.996517 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:52.996567 23581 dns.go:238] Pre-creating DNS records I1007 22:20:53.004520 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1007 22:20:53.009040 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.009139 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.009199 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.009253 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.051292 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.244276 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.246518 23581 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1007 22:20:53.246720 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1007 22:20:53.246965 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1007 22:20:53.247107 23581 api.go:53] PageSize not implemented I1007 22:20:53.247075 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:53.247189 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1007 22:20:53.247353 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1007 22:20:53.247450 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1007 22:20:53.247609 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1007 22:20:53.247716 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1007 22:20:53.247879 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.247994 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:53.248026 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:53.248082 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.248131 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1007 22:20:53.249795 23581 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1007 22:20:53.249933 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.250072 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1007 22:20:53.250156 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:53.250128 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.250282 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:53.250260 23581 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1007 22:20:53.250339 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:53.250366 23581 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:53.250249 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1007 22:20:53.250536 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.250794 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1007 22:20:53.250864 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.250963 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.251096 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.251270 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:53.251341 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1007 22:20:53.251525 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.251702 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.251849 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.251983 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.252132 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1007 22:20:53.252283 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.252317 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.252452 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:53.252541 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:53.255459 23581 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1007 22:20:53.255701 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.255835 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.255927 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:53.256011 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:53.256031 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:53.256069 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:53.256104 23581 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:53.256144 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1007 22:20:53.256176 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1007 22:20:53.256208 23581 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1007 22:20:53.256184 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.256407 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:53.256546 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.256642 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.256836 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:53.257032 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.257171 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:53.257301 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.257469 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.260040 23581 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1007 22:20:53.260134 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:53.260224 23581 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1007 22:20:53.260254 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1007 22:20:53.260281 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:53.260360 23581 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1007 22:20:53.260498 23581 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1007 22:20:53.260827 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:53.260886 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:53.260924 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:53.260961 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:53.261000 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:53.261042 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:53.261077 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:53.261111 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:53.261145 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:53.261183 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:53.261223 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:53.261269 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:53.261305 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:53.261338 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:53.261377 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:53.261411 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:53.261450 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:53.261487 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:53.261525 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:53.261562 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:53.261602 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:53.261636 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:53.261676 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1007 22:20:53.261719 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:53.261754 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:53.261790 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:53.261828 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:53.261869 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:53.261910 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:53.264614 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:53.264647 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:53.264681 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.264741 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1007 22:20:53.264806 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.264866 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.264915 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.264970 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.265016 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.265063 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1007 22:20:53.265121 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.265194 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1007 22:20:53.265238 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W1007 22:20:53.265294 23581 api.go:53] PageSize not implemented I1007 22:20:53.265313 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:53.265339 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:53.265353 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1007 22:20:53.265407 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:53.265431 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:53.265449 23581 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I1007 22:20:53.265532 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.265559 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.265603 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:53.265638 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.265678 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:53.265730 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:53.265776 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } autoscaling-config:lt-2 ok I1007 22:20:53.266313 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I1007 22:20:53.266378 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } autoscaling-config:lt-1 ok I1007 22:20:53.266424 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } autoscaling-group:nodes.complex.example.com ok subnet:subnet-2 ok I1007 22:20:53.266422 23581 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok iam-instance-profile:masters.complex.example.com ok I1007 22:20:53.266490 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1007 22:20:53.266539 23581 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I1007 22:20:53.266578 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I1007 22:20:53.266530 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:53.266609 23581 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I1007 22:20:53.266467 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok I1007 22:20:53.266645 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:53.266733 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.266803 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W1007 22:20:53.267024 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:53.267047 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:53.267082 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:53.267138 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:53.267219 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:53.267264 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.267323 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 } ] } W1007 22:20:53.267647 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:53.267669 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:53.267702 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:53.267749 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1007 22:20:53.267819 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1007 22:20:53.267873 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1007 22:20:53.267866 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I1007 22:20:53.267961 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I1007 22:20:53.267982 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1007 22:20:53.268017 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I1007 22:20:53.268053 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I1007 22:20:53.268099 23581 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I1007 22:20:53.268112 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1007 22:20:53.268139 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I1007 22:20:53.268200 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:53.268283 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:53.268360 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:53.268663 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.73s) === RUN TestLifecycleExternalLB I1007 22:20:53.269934 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:53.270391 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:53.270470 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:53.270609 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:53.270707 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:53.270799 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:53.270855 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:53.270944 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:53.270999 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:53.271167 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I1007 22:20:53.300375 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:53.303654 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.303763 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.344946 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.441720 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.443070 23581 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1007 22:20:53.443246 23581 api.go:53] PageSize not implemented W1007 22:20:53.443346 23581 api.go:53] PageSize not implemented W1007 22:20:53.443397 23581 api.go:53] PageSize not implemented I1007 22:20:53.443315 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.444788 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1007 22:20:53.445036 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.445049 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I1007 22:20:53.445151 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:53.445210 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.445189 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1007 22:20:53.445371 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:53.445512 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:53.445582 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:53.445732 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:53.445796 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.445902 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I1007 22:20:53.446002 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I1007 22:20:53.446030 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.446159 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:53.446200 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:53.446239 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:53.446309 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.446359 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.446441 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.446644 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.443622 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:53.447278 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:53.447517 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:53.447810 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.448074 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.448388 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:53.448915 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.449046 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.449134 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:53.451350 23581 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1007 22:20:53.451545 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1007 22:20:53.451680 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1007 22:20:53.451546 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.451741 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1007 22:20:53.451775 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1007 22:20:53.451987 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:53.452047 23581 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1007 22:20:53.452085 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:53.452138 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.452193 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:53.452389 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.452403 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:53.452509 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:53.452557 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" } ] } I1007 22:20:53.452647 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:53.452586 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1007 22:20:53.452703 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1007 22:20:53.452922 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1007 22:20:53.453142 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.453352 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:53.453398 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.453600 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:53.453729 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:53.453579 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I1007 22:20:53.453793 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.453879 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:53.453949 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1007 22:20:53.454042 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:53.454096 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:53.453839 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1007 22:20:53.456576 23581 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1007 22:20:53.456801 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.456928 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.457004 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.457284 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" }] }] } I1007 22:20:53.457534 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1007 22:20:53.457835 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.458006 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.458130 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1007 22:20:53.458329 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.458529 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.458653 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.458779 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1007 22:20:53.458998 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.459221 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.459295 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.459566 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I1007 22:20:53.459916 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.460252 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I1007 22:20:53.460471 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1007 22:20:53.460646 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:53.460758 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:53.460837 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:53.460886 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:53.460947 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I1007 22:20:53.461156 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1007 22:20:53.461328 23581 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1007 22:20:53.461585 23581 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1007 22:20:53.461791 23581 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1007 22:20:53.461845 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.461884 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.461945 23581 dns.go:238] Pre-creating DNS records I1007 22:20:53.461978 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.468018 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:53.471313 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.471421 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.511906 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.612945 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.614730 23581 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1007 22:20:53.615222 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:53.615345 23581 api.go:53] PageSize not implemented I1007 22:20:53.615272 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.615436 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.615448 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:53.615334 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1007 22:20:53.615649 23581 api.go:53] PageSize not implemented I1007 22:20:53.615586 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W1007 22:20:53.615729 23581 api.go:53] PageSize not implemented I1007 22:20:53.615720 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:53.615747 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.615905 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.616020 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1007 22:20:53.615961 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.616063 23581 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:53.616207 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:53.616174 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.616329 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.616429 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:53.616460 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:53.616510 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.617850 23581 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1007 22:20:53.617987 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.618192 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1007 22:20:53.618109 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1007 22:20:53.618254 23581 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1007 22:20:53.618317 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.618396 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:53.618539 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1007 22:20:53.618711 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.618859 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:53.618900 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1007 22:20:53.619025 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1007 22:20:53.619195 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.623125 23581 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1007 22:20:53.623263 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.623377 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.623557 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:53.623804 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.623994 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.624345 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.624538 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.626706 23581 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1007 22:20:53.626962 23581 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1007 22:20:53.627096 23581 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1007 22:20:53.630366 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:53.630429 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:53.630468 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:53.630528 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:53.630563 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:53.630601 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:53.630636 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:53.630666 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:53.630701 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:53.630740 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:53.630778 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:53.630817 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:53.630858 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:53.630890 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:53.630923 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:53.630959 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:53.630991 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:53.631026 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:53.631062 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:53.631103 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:53.631136 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:53.631171 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:53.631205 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:53.631242 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:53.631279 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:53.631314 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:53.632676 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:53.632704 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:53.632742 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.632793 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1007 22:20:53.632849 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.632909 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.632958 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.633015 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.633057 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.633096 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1007 22:20:53.633144 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.633196 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1007 22:20:53.633237 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W1007 22:20:53.633293 23581 api.go:53] PageSize not implemented I1007 22:20:53.633318 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:53.633346 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:53.633365 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:53.633388 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:53.633406 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:53.633454 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.633487 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.633536 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:53.633573 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.633617 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:53.633658 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:53.633693 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok autoscaling-group:nodes.externallb.example.com ok I1007 22:20:53.634157 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.634260 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1007 22:20:53.634362 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I1007 22:20:53.634200 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I1007 22:20:53.634412 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1007 22:20:53.634502 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I1007 22:20:53.634254 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:53.634556 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:53.634574 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1007 22:20:53.634626 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:53.634697 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.634751 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:53.634869 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:53.634892 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:53.634923 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:53.634984 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:53.635023 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1007 22:20:53.635105 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:53.635190 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:53.635225 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I1007 22:20:53.635328 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I1007 22:20:53.635349 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1007 22:20:53.635386 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I1007 22:20:53.635424 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I1007 22:20:53.635472 23581 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I1007 22:20:53.635484 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1007 22:20:53.635513 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I1007 22:20:53.635579 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:53.635662 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:53.635736 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:53.636016 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I1007 22:20:53.636326 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:53.636566 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:53.636615 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:53.636659 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:53.636704 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:53.636747 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:53.636776 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:53.636816 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:53.636843 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:53.636905 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I1007 22:20:53.661095 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:53.664383 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.664477 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.705742 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.809418 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.810971 23581 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1007 22:20:53.811331 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:53.811477 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:53.811601 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I1007 22:20:53.811750 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1007 22:20:53.811738 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:53.811896 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:53.811964 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:53.811954 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I1007 22:20:53.812036 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:53.812065 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:53.812063 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1007 22:20:53.812107 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:53.812272 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.812351 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.812563 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.812771 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:53.812925 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:53.814266 23581 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1007 22:20:53.814494 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1007 22:20:53.814648 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1007 22:20:53.814556 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1007 22:20:53.814710 23581 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1007 22:20:53.814785 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1007 22:20:53.814824 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1007 22:20:53.815014 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1007 22:20:53.816556 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:53.816775 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:53.816943 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:53.816998 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1007 22:20:53.817179 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:53.817255 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:53.817376 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.817449 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:53.817563 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.817638 23581 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" } ] } I1007 22:20:53.821729 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1007 22:20:53.824223 23581 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1007 22:20:53.824427 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.824619 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.824782 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.825007 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" }] }] } I1007 22:20:53.825229 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.825483 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1007 22:20:53.825724 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.825869 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1007 22:20:53.826009 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.826147 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.826280 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I1007 22:20:53.826427 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.826558 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.826670 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.826744 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.826889 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.827016 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1007 22:20:53.827274 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I1007 22:20:53.829262 23581 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1007 22:20:53.829598 23581 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1007 22:20:53.829730 23581 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1007 22:20:53.829782 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.829828 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.829887 23581 dns.go:238] Pre-creating DNS records I1007 22:20:53.829918 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:53.836489 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:53.839805 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.839911 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.880550 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:53.978269 23581 iamrole.go:89] ListRoles: { } I1007 22:20:53.981659 23581 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1007 22:20:53.981845 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:53.982104 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:53.982016 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:53.982168 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:53.982370 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:53.982732 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.983124 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:53.984062 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:53.986422 23581 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1007 22:20:53.987060 23581 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1007 22:20:53.987174 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1007 22:20:53.987212 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1007 22:20:53.987531 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:53.987708 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1007 22:20:53.987838 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1007 22:20:53.988070 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:53.988140 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:53.992009 23581 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1007 22:20:53.992218 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.992464 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.992722 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.993021 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:53.995312 23581 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1007 22:20:53.995580 23581 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1007 22:20:53.995698 23581 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1007 22:20:53.998696 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:53.998785 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:53.998848 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:53.998912 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:53.998959 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:53.998995 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:53.999030 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:53.999064 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:53.999104 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:53.999142 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:53.999184 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:53.999232 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:53.999268 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:53.999301 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:53.999337 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:53.999373 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:53.999415 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:53.999450 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:53.999484 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:53.999521 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:53.999558 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:54.001005 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:54.001040 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:54.001080 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001129 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1007 22:20:54.001184 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001243 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001285 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001325 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001367 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001403 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1007 22:20:54.001441 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001483 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1007 22:20:54.001519 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W1007 22:20:54.001568 23581 api.go:53] PageSize not implemented I1007 22:20:54.001593 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:54.001621 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:54.001640 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:54.001665 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:54.001682 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:54.001734 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.001765 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.001816 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:54.001854 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.001901 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:54.001939 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:54.002306 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I1007 22:20:54.002376 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:54.002439 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.sharedsubnet.example.com ok I1007 22:20:54.002472 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } volume:vol-2 ok I1007 22:20:54.002402 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I1007 22:20:54.002576 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I1007 22:20:54.002501 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:54.002684 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.002650 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1007 22:20:54.002723 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:54.002749 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } autoscaling-config:lt-1 ok iam-instance-profile:masters.sharedsubnet.example.com ok I1007 22:20:54.002785 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.002853 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:54.003052 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.003072 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I1007 22:20:54.003148 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:54.003269 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I1007 22:20:54.003336 23581 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1007 22:20:54.003354 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1007 22:20:54.003389 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I1007 22:20:54.003426 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I1007 22:20:54.003508 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1007 22:20:54.003526 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1007 22:20:54.003563 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I1007 22:20:54.003903 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I1007 22:20:54.004172 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:54.004451 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:54.004501 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:54.004546 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:54.004591 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:54.004631 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:54.004659 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:54.004695 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:54.004720 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:54.004782 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I1007 22:20:54.036825 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:54.040553 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.040651 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.040719 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.082027 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.186728 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.189000 23581 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1007 22:20:54.189601 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:54.189755 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.189826 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.189840 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.189929 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1007 22:20:54.189968 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.190097 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:54.190160 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:54.190180 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:54.190214 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.190219 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.190297 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1007 22:20:54.190356 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.190407 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1007 22:20:54.190428 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.190583 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.190694 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:54.190733 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.190815 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:54.191636 23581 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1007 22:20:54.191765 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1007 22:20:54.191884 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1007 22:20:54.191949 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.192071 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1007 22:20:54.192136 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.192245 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1007 22:20:54.192282 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1007 22:20:54.192316 23581 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1007 22:20:54.192273 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.192364 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:54.192350 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.192408 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.192446 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:54.192548 23581 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I1007 22:20:54.192575 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.192641 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1007 22:20:54.192680 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.192802 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1007 22:20:54.192984 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.193056 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:54.193089 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1007 22:20:54.193179 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:54.193248 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1007 22:20:54.193348 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.193422 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:54.193466 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.193605 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:54.193663 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.193707 23581 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I1007 22:20:54.193806 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.193919 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:54.193977 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:54.194022 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:54.194071 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.194108 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.193973 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.197056 23581 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1007 22:20:54.197290 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W1007 22:20:54.197476 23581 api.go:53] PageSize not implemented W1007 22:20:54.197514 23581 api.go:53] PageSize not implemented I1007 22:20:54.197469 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1007 22:20:54.197598 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:54.197611 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1007 22:20:54.197801 23581 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1007 22:20:54.197784 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:54.197879 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1007 22:20:54.198015 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1007 22:20:54.198103 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1007 22:20:54.198014 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.198207 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.198240 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.198295 23581 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1007 22:20:54.198362 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.198395 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1007 22:20:54.198447 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1007 22:20:54.198412 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:54.198635 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:54.198830 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.198967 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I1007 22:20:54.199101 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.199287 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.199484 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I1007 22:20:54.199629 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.199773 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.199924 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.200029 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.200109 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.200323 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:54.200432 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" }] }] } I1007 22:20:54.200662 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.200816 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I1007 22:20:54.200956 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I1007 22:20:54.201078 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.201216 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.201370 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.201505 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1007 22:20:54.201635 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1007 22:20:54.201761 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.201865 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.202356 23581 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1007 22:20:54.202462 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.202636 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.202795 23581 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1007 22:20:54.202943 23581 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1007 22:20:54.203002 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.203036 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.203087 23581 dns.go:238] Pre-creating DNS records I1007 22:20:54.203114 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.213641 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:54.217897 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.218037 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.218104 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.259146 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.365095 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.367404 23581 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1007 22:20:54.367600 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:54.368025 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.368223 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:54.368357 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:54.368402 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:54.368454 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.368625 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.368700 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.370674 23581 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1007 22:20:54.370810 23581 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1007 22:20:54.370871 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1007 22:20:54.370886 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1007 22:20:54.371238 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1007 22:20:54.371090 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1007 22:20:54.371357 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.371426 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1007 22:20:54.371633 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1007 22:20:54.371890 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.371988 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1007 22:20:54.372215 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.372293 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:54.372403 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1007 22:20:54.372549 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.372606 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1007 22:20:54.372763 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:54.375944 23581 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1007 22:20:54.376204 23581 api.go:53] PageSize not implemented I1007 22:20:54.376240 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.376353 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.376390 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1007 22:20:54.376304 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W1007 22:20:54.376469 23581 api.go:53] PageSize not implemented I1007 22:20:54.376524 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.376504 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.376605 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1007 22:20:54.376649 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1007 22:20:54.376820 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.377011 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.377406 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.377602 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.379234 23581 executor.go:111] Tasks: 100 done / 107 total; 4 can run W1007 22:20:54.379340 23581 api.go:53] PageSize not implemented I1007 22:20:54.379353 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.379367 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.379453 23581 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I1007 22:20:54.379482 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W1007 22:20:54.379555 23581 api.go:53] PageSize not implemented I1007 22:20:54.379587 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.379817 23581 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1007 22:20:54.380033 23581 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1007 22:20:54.383519 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:54.383585 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:54.383622 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:54.383659 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:54.383691 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:54.383725 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:54.383757 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:54.383793 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:54.383849 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:54.383897 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:54.383935 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1007 22:20:54.383979 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:54.384013 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:54.384045 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:54.384079 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:54.384117 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:54.384155 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:54.384193 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:54.384228 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:54.384267 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:54.384306 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:54.384343 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:54.384378 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:54.384416 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:54.384453 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:54.386046 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:54.386077 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:54.386118 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386185 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1007 22:20:54.386266 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386328 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386374 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386422 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386470 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386512 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1007 22:20:54.386553 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386600 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1007 22:20:54.386638 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W1007 22:20:54.386686 23581 api.go:53] PageSize not implemented I1007 22:20:54.386705 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1007 22:20:54.386760 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:54.386788 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:54.386807 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:54.386832 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:54.386850 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:54.386901 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.386929 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.386981 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:54.387022 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.387068 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:54.387110 23581 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I1007 22:20:54.387719 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I1007 22:20:54.387783 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:nodes.private-shared-subnet.example.com ok I1007 22:20:54.387856 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I1007 22:20:54.387801 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.387927 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I1007 22:20:54.387766 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok autoscaling-group:bastion.private-shared-subnet.example.com ok I1007 22:20:54.387990 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.388055 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I1007 22:20:54.387875 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W1007 22:20:54.388093 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.388115 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1007 22:20:54.388088 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.388167 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I1007 22:20:54.388158 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.388265 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1007 22:20:54.388404 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.388426 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1007 22:20:54.388465 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:54.388529 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:54.388583 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:54.388639 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.388692 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:54.388779 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.388799 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I1007 22:20:54.388890 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.388943 23581 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1007 22:20:54.388895 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.388957 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1007 22:20:54.388995 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I1007 22:20:54.389029 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.388982 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1007 22:20:54.389109 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.389074 23581 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1007 22:20:54.389135 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:54.389177 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1007 22:20:54.389187 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.389223 23581 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I1007 22:20:54.389274 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.389331 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1007 22:20:54.389255 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1007 22:20:54.389376 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.389391 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:54.389345 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1007 22:20:54.389433 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I1007 22:20:54.389714 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedIP I1007 22:20:54.390038 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:54.390285 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:54.390331 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:54.390377 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:54.390427 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:54.390468 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:54.390531 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:54.390570 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:54.390597 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:54.390663 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I1007 22:20:54.423525 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:54.427117 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.427242 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.427307 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.468127 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.569639 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.571533 23581 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1007 22:20:54.572860 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:54.572901 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:54.573005 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:54.573065 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:54.573122 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.573098 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I1007 22:20:54.573189 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.573214 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1007 22:20:54.573367 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.573280 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.573471 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I1007 22:20:54.573708 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.573643 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.573791 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1007 22:20:54.573951 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1007 22:20:54.574345 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.574665 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:54.574797 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.575302 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:54.579276 23581 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1007 22:20:54.579421 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1007 22:20:54.579544 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1007 22:20:54.579557 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.579658 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1007 22:20:54.579712 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1007 22:20:54.579823 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1007 22:20:54.579792 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I1007 22:20:54.579863 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.579995 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:54.580052 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.580002 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.580093 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1007 22:20:54.580299 23581 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1007 22:20:54.580287 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.580380 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.580387 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:54.580538 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.580568 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1007 22:20:54.580792 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.580891 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.581006 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1007 22:20:54.581130 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.581229 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.581253 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.581341 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.581430 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:54.581473 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:54.581520 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.581421 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-12345678" } I1007 22:20:54.581559 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:54.581693 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1007 22:20:54.581679 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I1007 22:20:54.581770 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:54.581815 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1007 22:20:54.581835 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I1007 22:20:54.581903 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1007 22:20:54.581995 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:54.582038 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:54.582074 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:54.582121 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:54.582162 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:54.582221 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.582262 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.582325 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:54.582376 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:54.582416 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:54.582462 23581 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I1007 22:20:54.582400 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1007 22:20:54.584945 23581 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1007 22:20:54.585107 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.585212 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1007 22:20:54.585303 23581 api.go:53] PageSize not implemented W1007 22:20:54.585345 23581 api.go:53] PageSize not implemented I1007 22:20:54.585382 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:54.585457 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1007 22:20:54.585607 23581 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1007 22:20:54.585691 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1007 22:20:54.585653 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" }] }] } I1007 22:20:54.585757 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1007 22:20:54.585818 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1007 22:20:54.585882 23581 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1007 22:20:54.585955 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1007 22:20:54.585989 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1007 22:20:54.585953 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.586048 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1007 22:20:54.586122 23581 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1007 22:20:54.586170 23581 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1007 22:20:54.586158 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I1007 22:20:54.586304 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.586438 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.586589 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.586662 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.586812 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:54.586922 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.587004 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1007 22:20:54.587080 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1007 22:20:54.587175 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.587337 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.587430 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.587575 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:54.587614 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:54.587654 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I1007 22:20:54.587797 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:54.587893 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1007 22:20:54.588007 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:54.588128 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.588255 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I1007 22:20:54.588381 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1007 22:20:54.588506 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1007 22:20:54.588551 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1007 22:20:54.588585 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:54.588708 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.588938 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.589105 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.589262 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.589402 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.589554 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.589667 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.589743 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.589931 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I1007 22:20:54.590117 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1007 22:20:54.590279 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.590390 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.590476 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:54.590605 23581 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I1007 22:20:54.592902 23581 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1007 22:20:54.593007 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.593149 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.593242 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.593326 23581 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:54.593565 23581 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1007 22:20:54.593756 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:54.593835 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:54.593899 23581 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1007 22:20:54.594003 23581 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1007 22:20:54.594082 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.594245 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1007 22:20:54.594340 23581 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1007 22:20:54.594385 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.594444 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.594505 23581 dns.go:238] Pre-creating DNS records I1007 22:20:54.594540 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.601221 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1007 22:20:54.604939 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.605034 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.605119 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.645621 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.753421 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.755782 23581 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1007 22:20:54.756141 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1007 22:20:54.756247 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1007 22:20:54.756283 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1007 22:20:54.756358 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:54.756315 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.756502 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.756805 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.756880 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.761104 23581 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1007 22:20:54.761340 23581 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1007 22:20:54.761627 23581 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1007 22:20:54.761296 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1007 22:20:54.761964 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1007 22:20:54.761990 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.762491 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.763364 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:54.763418 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1007 22:20:54.763597 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.763650 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I1007 22:20:54.764405 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1007 22:20:54.764604 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1007 22:20:54.764737 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.764850 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1007 22:20:54.764931 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1007 22:20:54.765053 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.765090 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1007 22:20:54.765129 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:54.765186 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.765259 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.768655 23581 executor.go:111] Tasks: 71 done / 116 total; 36 can run W1007 22:20:54.768812 23581 api.go:53] PageSize not implemented I1007 22:20:54.768840 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1007 22:20:54.768971 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1007 22:20:54.769033 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } W1007 22:20:54.769163 23581 api.go:53] PageSize not implemented I1007 22:20:54.769128 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.769198 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1007 22:20:54.769320 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1007 22:20:54.769357 23581 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1007 22:20:54.769396 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.769532 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.769698 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1007 22:20:54.769859 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:54.770007 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.770286 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.770600 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.770851 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.771001 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.771212 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.771383 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1007 22:20:54.774184 23581 executor.go:111] Tasks: 107 done / 116 total; 5 can run W1007 22:20:54.774294 23581 api.go:53] PageSize not implemented I1007 22:20:54.774304 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.774325 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1007 22:20:54.774403 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:54.774467 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1007 22:20:54.774608 23581 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" W1007 22:20:54.774561 23581 api.go:53] PageSize not implemented I1007 22:20:54.774659 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1007 22:20:54.774878 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1007 22:20:54.774990 23581 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1007 22:20:54.775098 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1007 22:20:54.775271 23581 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1007 22:20:54.779147 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:54.779214 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:54.779255 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:54.779291 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:54.779331 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:54.779369 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:54.779405 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:54.779447 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:54.779521 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:54.779575 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:54.779617 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1007 22:20:54.779661 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1007 22:20:54.779700 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:54.779737 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:54.779772 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:54.779811 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:54.779843 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1007 22:20:54.779880 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:54.779917 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:54.779953 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1007 22:20:54.779991 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1007 22:20:54.780028 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1007 22:20:54.780067 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:54.780110 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:54.780145 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1007 22:20:54.780183 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:54.780217 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:54.780251 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:54.780289 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:54.780327 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:54.781894 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:54.781927 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:54.781966 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782029 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1007 22:20:54.782104 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782163 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782206 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782246 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782288 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782333 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1007 22:20:54.782388 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782445 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:54.782476 23581 natgateway.go:117] DescribeNatGateways: { } I1007 22:20:54.782520 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1007 22:20:54.782560 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W1007 22:20:54.782608 23581 api.go:53] PageSize not implemented I1007 22:20:54.782626 23581 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1007 22:20:54.782682 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:54.782711 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:54.782730 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:54.782754 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:54.782772 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1007 22:20:54.782825 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.782856 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:54.782899 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:54.782938 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.782987 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:54.783036 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:54.783070 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1007 22:20:54.783107 23581 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1007 22:20:54.783147 23581 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:bastion.private-shared-ip.example.com ok I1007 22:20:54.783698 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1007 22:20:54.783780 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I1007 22:20:54.783775 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:54.783863 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.783876 23581 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok nat-gateway:nat-1 ok autoscaling-group:nodes.private-shared-ip.example.com ok autoscaling-config:lt-1 ok I1007 22:20:54.783932 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.783961 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1007 22:20:54.784038 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I1007 22:20:54.783938 23581 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I1007 22:20:54.784104 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I1007 22:20:54.783854 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok I1007 22:20:54.784157 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1007 22:20:54.784245 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I1007 22:20:54.784077 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:54.784363 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.784391 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1007 22:20:54.784430 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1007 22:20:54.784494 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.784552 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1007 22:20:54.784659 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.784679 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:54.784729 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1007 22:20:54.784819 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:54.784911 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.784926 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1007 22:20:54.785046 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I1007 22:20:54.785057 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } subnet:subnet-2 ok I1007 22:20:54.785144 23581 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1007 22:20:54.785139 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:54.785170 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1007 22:20:54.785231 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1007 22:20:54.785264 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I1007 22:20:54.785329 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.785293 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1007 22:20:54.785399 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com W1007 22:20:54.785404 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.785421 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1007 22:20:54.785415 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1007 22:20:54.785468 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I1007 22:20:54.785488 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.785506 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.785638 23581 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1007 22:20:54.785565 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1007 22:20:54.785673 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:54.785690 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:54.785654 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1007 22:20:54.785744 23581 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I1007 22:20:54.785813 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:54.785896 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1007 22:20:54.786196 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I1007 22:20:54.786530 23581 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:20:54.786798 23581 internetgateways.go:66] CreateInternetGateway: { } I1007 22:20:54.786846 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:20:54.786888 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:20:54.786940 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:20:54.786982 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:20:54.787013 23581 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:20:54.787053 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:20:54.787080 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:20:54.787141 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I1007 22:20:54.811957 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:54.815254 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:54.815345 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:54.856612 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:54.981846 23581 iamrole.go:89] ListRoles: { } I1007 22:20:54.983322 23581 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1007 22:20:54.983726 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:54.983839 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.983992 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:54.984198 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:54.984140 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.984301 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] } I1007 22:20:54.984455 23581 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:54.984405 23581 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:54.984529 23581 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1007 22:20:54.984555 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:54.984612 23581 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:54.986257 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.987234 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.988155 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.988297 23581 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:54.988607 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.988849 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1007 22:20:54.988927 23581 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1007 22:20:54.988996 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:54.989099 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:54.990087 23581 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1007 22:20:54.991009 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:54.991108 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:54.992476 23581 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1007 22:20:54.992861 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:54.993017 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1007 22:20:54.993137 23581 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1007 22:20:54.993120 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.993178 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:54.993306 23581 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:54.993371 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1007 22:20:54.993385 23581 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I1007 22:20:54.993583 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:54.993644 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.993872 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1007 22:20:54.993809 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:54.994035 23581 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1007 22:20:54.993978 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1007 22:20:54.994368 23581 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1007 22:20:54.994449 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:54.994671 23581 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:20:54.994745 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:54.994987 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:54.995096 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:54.995309 23581 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1007 22:20:54.995487 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:54.995546 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:54.995595 23581 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:54.995767 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:54.995864 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:54.995966 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:54.996033 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:54.996125 23581 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1007 22:20:54.996169 23581 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:54.999367 23581 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1007 22:20:54.999507 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.999672 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1007 22:20:54.999721 23581 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1007 22:20:54.999775 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:54.999998 23581 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:55.000103 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.000234 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.000455 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.000790 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.000952 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.001141 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:55.001216 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.001360 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.001504 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.001650 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I1007 22:20:55.001787 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.001922 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.002061 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.002167 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.002244 23581 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1007 22:20:55.002376 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.002562 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.002711 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1007 22:20:55.002850 23581 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.003004 23581 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1007 22:20:55.004433 23581 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1007 22:20:55.004674 23581 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1007 22:20:55.004911 23581 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1007 22:20:55.004973 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:55.005027 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:55.005089 23581 dns.go:238] Pre-creating DNS records I1007 22:20:55.005119 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:55.011122 23581 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1007 22:20:55.014523 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.014616 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.056452 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:55.187384 23581 iamrole.go:89] ListRoles: { } I1007 22:20:55.189171 23581 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1007 22:20:55.189479 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.189847 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.190005 23581 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1007 22:20:55.190094 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.190298 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:20:55.190360 23581 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:20:55.190439 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.190671 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:55.190754 23581 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1007 22:20:55.190879 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:55.193407 23581 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1007 22:20:55.193681 23581 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1007 22:20:55.193766 23581 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1007 22:20:55.193632 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1007 22:20:55.193995 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.194394 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:55.194576 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.194913 23581 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:20:55.195231 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:55.196659 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1007 22:20:55.196849 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.196960 23581 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:20:55.197890 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:55.201764 23581 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1007 22:20:55.202102 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:55.202375 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.202565 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.202721 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:55.202902 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.203051 23581 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1007 22:20:55.203198 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1007 22:20:55.204692 23581 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1007 22:20:55.204961 23581 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1007 22:20:55.205178 23581 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1007 22:20:55.208565 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1007 22:20:55.208627 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1007 22:20:55.208662 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1007 22:20:55.208694 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1007 22:20:55.208728 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1007 22:20:55.208765 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1007 22:20:55.208798 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1007 22:20:55.208828 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1007 22:20:55.208861 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1007 22:20:55.208905 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1007 22:20:55.208942 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1007 22:20:55.208979 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1007 22:20:55.209019 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1007 22:20:55.209051 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1007 22:20:55.209083 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1007 22:20:55.209121 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1007 22:20:55.209153 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:20:55.209187 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1007 22:20:55.209232 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:20:55.209297 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1007 22:20:55.209359 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1007 22:20:55.209416 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1007 22:20:55.209478 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1007 22:20:55.209550 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1007 22:20:55.209625 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:20:55.209686 23581 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1007 22:20:55.211436 23581 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1007 22:20:55.211466 23581 keypairs.go:95] DescribeKeyPairs: { } I1007 22:20:55.211505 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211559 23581 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211618 23581 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211678 23581 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211725 23581 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211779 23581 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211824 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211864 23581 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211916 23581 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.211973 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1007 22:20:55.212016 23581 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W1007 22:20:55.212075 23581 api.go:53] PageSize not implemented I1007 22:20:55.212097 23581 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1007 22:20:55.212125 23581 loadbalancers.go:34] PageSize not implemented I1007 22:20:55.212146 23581 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1007 22:20:55.212170 23581 targetgroups.go:35] PageSize not implemented I1007 22:20:55.212189 23581 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1007 22:20:55.212244 23581 zones.go:93] ListHostedZonesPages { } I1007 22:20:55.212274 23581 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1007 22:20:55.212326 23581 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1007 22:20:55.212366 23581 iamrole.go:89] ListRoles: { } I1007 22:20:55.212413 23581 internetgateways.go:101] DescribeInternetGateways: { } I1007 22:20:55.212457 23581 routetable.go:53] DescribeRouteTables: { } I1007 22:20:55.212491 23581 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I1007 22:20:55.213263 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1007 22:20:55.213286 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:55.213433 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I1007 22:20:55.213360 23581 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok I1007 22:20:55.213359 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1007 22:20:55.213667 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:55.213689 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1007 22:20:55.213525 23581 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:55.213737 23581 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I1007 22:20:55.213723 23581 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1007 22:20:55.213808 23581 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1007 22:20:55.213881 23581 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1007 22:20:55.214007 23581 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1007 22:20:55.214075 23581 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1007 22:20:55.214196 23581 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1007 22:20:55.214211 23581 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1007 22:20:55.214244 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1007 22:20:55.214303 23581 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I1007 22:20:55.214394 23581 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1007 22:20:55.214445 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:55.214539 23581 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1007 22:20:55.214559 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1007 22:20:55.214596 23581 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I1007 22:20:55.214641 23581 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:55.214688 23581 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1007 22:20:55.214701 23581 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1007 22:20:55.214731 23581 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I1007 22:20:55.214797 23581 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1007 22:20:55.214875 23581 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1007 22:20:55.214944 23581 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1007 22:20:55.215219 23581 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.43s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 62.225s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.010s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I1007 22:19:32.501701 14761 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.047s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.067s === RUN TestPodController I1007 22:19:37.902037 19341 pod.go:59] starting pod controller I1007 22:19:37.902393 19341 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.070s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.071s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00018ddd0} route53_test.go:93: Got zone list: [0xc00018de00] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00018ded0} route53_test.go:93: Got zone list: [0xc00018df00] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00018dfa0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003a12f0 0xc0001940a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c2100} route53_test.go:93: Got zone list: [0xc0003c2130] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc0001940c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c2230} route53_test.go:93: Got zone list: [0xc0003c2260] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc0001940e0} route53_test.go:187: Successfully added resource record set: {0xc0001f8600 0xc000194100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c24e0} route53_test.go:93: Got zone list: [0xc0003c2510] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194188} route53_test.go:199: Successfully added resource record set: {0xc0001f8780 0xc0001941a8} route53_test.go:118: Got 1 recordsets: [0xc0003c26b0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c27e0} route53_test.go:93: Got zone list: [0xc0003c2810] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194240} route53_test.go:221: Successfully added resource record set: {0xc0001f8900 0xc000194260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0001f8900 0xc000194260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c2b70} route53_test.go:93: Got zone list: [0xc0003c2ba0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194318} route53_test.go:246: Successfully removed resource set {0xc0001f8b00 0xc000194338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c2e20} route53_test.go:93: Got zone list: [0xc0003c2e50] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc0001943c0} route53_test.go:264: Successfully removed resource set {0xc0001f8c80 0xc0001943e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c3100} route53_test.go:93: Got zone list: [0xc0003c3130] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194470} commontests.go:192: Successfully added resource record set: {0xc0001f8e00 0xc000194490} commontests.go:45: Correctly replaced resource record {0xc0001f8e00 0xc000194490} -> {0xc0001f8f00 0xc0001944e0} commontests.go:142: Got 1 recordsets: [0xc0003c3440] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c3610} route53_test.go:93: Got zone list: [0xc0003c3640] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc000194590} commontests.go:192: Successfully added resource record set: {0xc0001f9180 0xc0001945b0} commontests.go:70: Correctly replaced resource record {0xc0001f9180 0xc0001945b0} -> {0xc0001f9280 0xc000194600} commontests.go:142: Got 1 recordsets: [0xc0003c3950] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c3b70} route53_test.go:93: Got zone list: [0xc0003c3ba0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc0001946c0} commontests.go:192: Successfully added resource record set: {0xc0001f9500 0xc0001946e0} commontests.go:142: Got 2 recordsets: [0xc0003c3e10 0xc0003c3e30] commontests.go:142: Got 2 recordsets: [0xc0003c3ea0 0xc0003c3ec0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003d4060} route53_test.go:93: Got zone list: [0xc0003d4090] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a1110 0xc0001947e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.068s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.043s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c120} clouddns_test.go:71: Got zone list: [0xc0000103c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000010360} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c220} clouddns_test.go:71: Got zone list: [0xc000010510] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc0000104b0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c320} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0004090c0 0xc0000107c8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c3e0} clouddns_test.go:71: Got zone list: [0xc000010930] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc0000108d0} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c500} clouddns_test.go:71: Got zone list: [0xc000010f60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000010ee8} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c640} clouddns_test.go:71: Got zone list: [0xc000011200] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc0000111a0} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c7a0} clouddns_test.go:71: Got zone list: [0xc000011698] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000011590} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011c900} clouddns_test.go:71: Got zone list: [0xc000011818] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc0000117e8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011ca40} clouddns_test.go:71: Got zone list: [0xc000011980] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000011938} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011cba0} clouddns_test.go:71: Got zone list: [0xc000011b90] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000011b60} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011cd00} clouddns_test.go:71: Got zone list: [0xc000011e48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000011e18} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011ce80} clouddns_test.go:71: Got zone list: [0xc000652240] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc000652210} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a048 0xc00011d060} clouddns_test.go:71: Got zone list: [0xc000652600] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000408f80 0xc0006525d0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.081s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.055s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.207s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I1007 22:19:53.121672 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.121875 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.121916 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.121947 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.121992 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.122032 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.122044 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.122069 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.122100 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.128306 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I1007 22:19:53.131332 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.131478 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.131512 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.131537 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.131561 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.131581 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.131594 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.131612 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.131625 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.132492 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I1007 22:19:53.135399 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.135507 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.135535 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.135556 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.135587 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.135605 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.135618 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.135633 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.135651 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.136518 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.137938 23610 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I1007 22:19:53.137952 23610 assetstore.go:111] bin/containerd I1007 22:19:53.137959 23610 assetstore.go:111] bin/containerd-shim I1007 22:19:53.137964 23610 assetstore.go:111] bin/containerd-shim-runc-v1 I1007 22:19:53.137970 23610 assetstore.go:111] bin/containerd-shim-runc-v2 I1007 22:19:53.137981 23610 assetstore.go:111] bin/containerd-stress I1007 22:19:53.137990 23610 assetstore.go:111] bin/ctr I1007 22:19:53.138024 23610 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I1007 22:19:53.138035 23610 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I1007 22:19:53.142287 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.142429 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.142456 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.142488 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.142515 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.142534 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.142547 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.142566 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.142580 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.143528 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.144678 23610 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I1007 22:19:53.145459 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.145553 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.145573 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.145592 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.145613 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.145671 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.145686 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.145717 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.145730 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.146528 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.147758 23610 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I1007 22:19:53.148308 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.148442 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.148468 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.148488 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.148511 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.148548 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.148561 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.148578 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.148592 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.149449 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.150754 23610 assetstore.go:108] Matching assets for "^docker/": I1007 22:19:53.150769 23610 assetstore.go:111] docker/containerd I1007 22:19:53.150776 23610 assetstore.go:111] docker/containerd-shim I1007 22:19:53.150782 23610 assetstore.go:111] docker/ctr I1007 22:19:53.150787 23610 assetstore.go:111] docker/docker I1007 22:19:53.150793 23610 assetstore.go:111] docker/docker-init I1007 22:19:53.150798 23610 assetstore.go:111] docker/docker-proxy I1007 22:19:53.150803 23610 assetstore.go:111] docker/dockerd I1007 22:19:53.150809 23610 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I1007 22:19:53.153199 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.153345 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.153376 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.153398 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.153425 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.153481 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.153496 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.153515 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.153529 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.154453 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.155680 23610 assetstore.go:108] Matching assets for "^docker/": I1007 22:19:53.155694 23610 assetstore.go:111] docker/containerd I1007 22:19:53.155701 23610 assetstore.go:111] docker/containerd-shim I1007 22:19:53.155706 23610 assetstore.go:111] docker/ctr I1007 22:19:53.155711 23610 assetstore.go:111] docker/docker I1007 22:19:53.155716 23610 assetstore.go:111] docker/docker-init I1007 22:19:53.155722 23610 assetstore.go:111] docker/docker-proxy I1007 22:19:53.155728 23610 assetstore.go:111] docker/dockerd I1007 22:19:53.155734 23610 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.00s) === RUN TestDockerBuilder_SkipInstall I1007 22:19:53.158077 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.158224 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.158252 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.158275 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.158309 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.158381 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.158403 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.158433 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.158453 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.159818 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.161443 23610 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I1007 22:19:53.161649 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.161768 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.161801 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.161823 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.161846 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.161886 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.161901 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.161919 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.161932 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.162766 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:53.163935 23610 assetstore.go:108] Matching assets for "^docker/": I1007 22:19:53.163949 23610 assetstore.go:111] docker/docker I1007 22:19:53.163955 23610 assetstore.go:111] docker/docker-containerd I1007 22:19:53.163963 23610 assetstore.go:111] docker/docker-containerd-ctr I1007 22:19:53.163968 23610 assetstore.go:111] docker/docker-containerd-shim I1007 22:19:53.163974 23610 assetstore.go:111] docker/docker-init I1007 22:19:53.163981 23610 assetstore.go:111] docker/docker-proxy I1007 22:19:53.163986 23610 assetstore.go:111] docker/docker-runc I1007 22:19:53.163991 23610 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I1007 22:19:53.167829 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.167967 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.167994 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.168015 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.168038 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.168057 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.168072 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.168090 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.168110 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.170191 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I1007 22:19:53.173882 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.174028 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.174057 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.174083 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.174111 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.174136 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.174149 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.174167 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.174181 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.175718 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I1007 22:19:53.177741 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.177856 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.177877 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.177897 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.177920 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.177938 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.177953 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.177970 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.177982 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.179284 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I1007 22:19:53.184281 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.184420 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.184452 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.184476 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.184502 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.184524 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.184539 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.184558 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.184572 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.186010 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I1007 22:19:53.193178 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.193321 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.193348 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.193371 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.193397 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.193419 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.193433 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.193455 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.193470 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.195035 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I1007 22:19:53.201704 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.201936 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.201977 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.202011 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.202044 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.202074 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.202094 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.202120 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.202147 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.204138 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I1007 22:19:53.211664 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.211814 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.211851 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.211881 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.211917 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.211945 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.211966 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.211993 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.212010 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.214127 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I1007 22:19:53.223612 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:53.223776 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.223842 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:53.223899 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:53.223935 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:53.223965 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:53.223986 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:53.224023 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:53.224042 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:53.226188 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.51s) === RUN TestKubeAPIServerBuilderARM64 I1007 22:19:54.735922 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.736111 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.736149 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.736183 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.736223 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.736253 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.736273 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.736296 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.736317 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.740506 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I1007 22:19:54.816424 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.816609 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.816647 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.816679 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.816711 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.816740 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.816761 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.816785 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.816808 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.818651 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I1007 22:19:54.824889 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.825058 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.825093 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.825119 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.825152 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.825180 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.825198 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.825220 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.825237 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.826984 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I1007 22:19:54.900343 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.900552 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.900591 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.900621 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.900656 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.900691 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.900709 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.900732 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.900750 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.904200 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I1007 22:19:54.981181 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.981392 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.981435 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.981480 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.981530 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.981565 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.981594 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.981627 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.981651 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.984369 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I1007 22:19:54.991488 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:54.991700 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:54.991746 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:54.991790 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:54.991835 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:54.991875 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:54.991903 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:54.991939 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:54.991964 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:54.994834 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.08s) === RUN TestKubeProxyBuilderARM64 I1007 22:19:55.067702 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.067973 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.068042 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.068092 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.068139 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.068176 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.068202 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.068236 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.068259 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.070700 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I1007 22:19:55.139447 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.139664 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.139712 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.139755 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.139826 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.139868 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.139897 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.139935 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.139961 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.142789 23610 zones.go:93] ListHostedZonesPages { } W1007 22:19:55.149321 23610 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I1007 22:19:55.152509 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.152786 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.152838 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.152896 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.152976 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.153044 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.153101 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.153148 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.153203 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.156185 23610 zones.go:93] ListHostedZonesPages { } W1007 22:19:55.238203 23610 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.09s) === RUN TestKubeSchedulerBuilderARM64 I1007 22:19:55.241244 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.241480 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.241529 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.241577 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.241623 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.241666 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.241696 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.241739 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.241763 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.244648 23610 zones.go:93] ListHostedZonesPages { } W1007 22:19:55.314554 23610 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I1007 22:19:55.317328 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.317545 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.317589 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.317635 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.317680 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.317719 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.317748 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.317786 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.317812 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.320264 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:55.322696 23610 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I1007 22:19:55.325500 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.325663 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.325718 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.325750 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.325791 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.325825 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.325851 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.325885 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.325907 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.327678 23610 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I1007 22:19:55.332063 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.332339 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.332541 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.332640 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.332713 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.333062 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.333084 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.333107 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.333123 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.334417 23610 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I1007 22:19:55.337620 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.337751 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.337778 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.337802 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.337828 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.337849 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.337865 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.337885 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.337898 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.339402 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:55.340934 23610 assetstore.go:108] Matching assets for "protokube$": I1007 22:19:55.340953 23610 assetstore.go:111] /opt/kops/bin/protokube I1007 22:19:55.340963 23610 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I1007 22:19:55.340984 23610 assetstore.go:108] Matching assets for "channels$": I1007 22:19:55.340993 23610 assetstore.go:111] /opt/kops/bin/channels I1007 22:19:55.341001 23610 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I1007 22:19:55.342349 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.342466 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.342493 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.342516 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.342579 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.342602 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.342620 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.342639 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.342654 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.344141 23610 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I1007 22:19:55.346135 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.346267 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.346292 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.346321 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.346348 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.346366 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.346384 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.346404 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.346417 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.347926 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:55.349411 23610 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I1007 22:19:55.349806 23610 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:55.349923 23610 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:55.349947 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:55.349968 23610 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:55.349992 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:55.350014 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:55.350030 23610 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:55.350051 23610 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:55.350064 23610 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:55.351590 23610 zones.go:93] ListHostedZonesPages { } I1007 22:19:55.353064 23610 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.291s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.100s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.073s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.053s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.057s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E1007 22:19:34.951617 16582 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E1007 22:19:34.951780 16582 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.046s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I1007 22:19:46.031825 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.031975 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032064 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032131 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032203 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I1007 22:19:46.032558 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032629 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032690 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032751 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032823 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032883 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.032950 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033010 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033078 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033145 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033203 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033269 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033370 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033429 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033485 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033576 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033637 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033704 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1007 22:19:46.033766 21652 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I1007 22:19:46.034089 21652 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I1007 22:19:46.034191 21652 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.184s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I1007 22:19:38.947686 19701 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1007 22:19:38.948555 19701 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1007 22:19:38.949324 19701 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1007 22:19:38.949814 19701 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1007 22:19:38.951007 19701 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1007 22:19:38.951350 19701 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1007 22:19:38.952614 19701 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1007 22:19:38.952736 19701 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1007 22:19:38.953561 19701 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I1007 22:19:38.953606 19701 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I1007 22:19:38.953887 19701 health.go:33] status conditions not found for /v1, Kind=Namespace I1007 22:19:38.954116 19701 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I1007 22:19:38.954162 19701 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I1007 22:19:38.954392 19701 health.go:33] status conditions not found for /v1, Kind=ConfigMap I1007 22:19:38.954602 19701 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I1007 22:19:38.954655 19701 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I1007 22:19:38.955345 19701 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I1007 22:19:38.955392 19701 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.02s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.055s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W1007 22:19:36.181286 17699 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.09s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W1007 22:19:36.182644 17699 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W1007 22:19:36.183869 17699 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W1007 22:19:36.185070 17699 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W1007 22:19:36.186126 17699 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W1007 22:19:36.187192 17699 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W1007 22:19:36.188271 17699 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E1007 22:19:36.189151 17699 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E1007 22:19:36.189168 17699 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E1007 22:19:36.189178 17699 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E1007 22:19:36.189190 17699 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.185s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.084s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.143s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.046s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.02s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.01s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.162s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.068s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.013s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.062s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.080s === RUN TestFlagToFalse I1007 22:19:37.894747 19368 featureflag_test.go:32] Created flag Unittest1 I1007 22:19:37.894818 19368 featureflag.go:160] FeatureFlag "UnitTest1"=false I1007 22:19:37.894837 19368 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I1007 22:19:37.894926 19368 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.027s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.106s === RUN TestRenderInstanceGroupZones W1007 22:19:40.317511 20046 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.051s === RUN TestWarmPoolOnlyRoll I1007 22:20:04.647841 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.647903 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.647913 23944 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I1007 22:20:04.648241 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.648270 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.649474 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.649499 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.654688 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.654718 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.654732 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.654761 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.656612 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.657337 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.657370 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.658558 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.658575 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.663863 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.663906 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:04.664680 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.664718 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.665945 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.665962 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.671201 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.671241 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.671282 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.671290 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.672222 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.672987 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.673026 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.674208 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.674226 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.679463 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.679514 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.680321 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.680361 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.681555 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.681573 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.686821 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.686873 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:04.687792 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.687832 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.689030 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.689048 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.694307 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.694341 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.694362 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.694377 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:04.695315 23944 instancegroups.go:402] Draining the node: "node-2a.local". I1007 22:20:04.696059 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:04.696095 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.697302 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.697340 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.702624 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.702674 23944 instancegroups.go:402] Draining the node: "node-2b.local". I1007 22:20:04.703505 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:04.703545 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.704735 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.704771 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.710033 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.710083 23944 instancegroups.go:402] Draining the node: "node-2c.local". I1007 22:20:04.710942 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:04.710982 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.712183 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.712219 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.717461 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.717495 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I1007 22:20:04.717930 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.717971 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.719161 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.719220 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.719267 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.719285 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.719332 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.720548 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.720600 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.720615 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.720643 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.721838 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.721873 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.721921 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.721939 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.721964 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.723157 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.723209 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.723227 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.723254 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.724470 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.724522 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.724537 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.724564 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.725781 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.725809 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.725855 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.725870 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:04.725896 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.727098 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.727149 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.727182 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:04.727211 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.728389 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1007 22:20:04.728439 23944 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1007 22:20:04.728454 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:04.728481 23944 instancegroups.go:438] waiting for 1ms after terminating instance W1007 22:20:04.729683 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1007 22:20:04.729714 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I1007 22:20:04.730001 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.730036 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.731242 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.731285 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.731307 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.731356 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.731382 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.731397 23944 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.731410 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.732128 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.732899 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.732939 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.734122 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.734165 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.734193 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.734234 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:04.735070 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.735110 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.736305 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.736345 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.736366 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.736387 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.736407 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.736421 23944 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.736435 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.737503 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.738265 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.738303 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.739463 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.739504 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.739526 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.739566 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.740382 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.740421 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.741587 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.741629 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.741650 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.741692 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:04.742543 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.742582 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.743749 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.743789 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.743810 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.743844 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.743866 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.743885 23944 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.743899 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:04.744856 23944 instancegroups.go:402] Draining the node: "node-2a.local". I1007 22:20:04.745625 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:04.745664 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.746826 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.746858 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.746876 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.746913 23944 instancegroups.go:402] Draining the node: "node-2b.local". I1007 22:20:04.747693 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:04.747732 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.748926 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.748967 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.748987 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.749026 23944 instancegroups.go:402] Draining the node: "node-2c.local". I1007 22:20:04.749997 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:04.750036 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.751196 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.751235 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1007 22:20:04.751254 23944 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1007 22:20:04.751271 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I1007 22:20:04.751521 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I1007 22:20:04.751769 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.751802 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.752956 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.752992 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.758211 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.758263 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.758283 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.758295 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.758981 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.759876 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.759915 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.761110 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.761147 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.766380 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.766425 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:04.767187 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.767224 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.768430 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.768465 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.773709 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.773738 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.773757 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.773770 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.774682 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.775407 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.775445 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.776648 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.776684 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.781905 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.781943 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.782788 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.782826 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.784007 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.784043 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.789285 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.789325 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:04.790070 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.790107 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.791319 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.791362 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.796594 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.796618 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.796637 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.796649 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:04.797549 23944 instancegroups.go:402] Draining the node: "node-2a.local". I1007 22:20:04.798322 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:04.798361 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.799566 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.799602 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.804833 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.804872 23944 instancegroups.go:402] Draining the node: "node-2b.local". I1007 22:20:04.805635 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:04.805673 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.806863 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.806899 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.812120 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.812159 23944 instancegroups.go:402] Draining the node: "node-2c.local". I1007 22:20:04.812915 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:04.812953 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.814136 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.814173 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.819415 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.819440 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I1007 22:20:04.819552 23944 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I1007 22:20:04.819950 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.819981 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.821172 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.821216 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.821236 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I1007 22:20:04.821619 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.821654 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.822805 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.822834 23944 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1007 22:20:04.822854 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I1007 22:20:04.823073 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.823103 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.823119 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I1007 22:20:04.823264 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.823289 23944 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1007 22:20:04.823305 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I1007 22:20:04.823514 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.823544 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.824741 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.824785 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.830033 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.830080 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.830102 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.830114 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.830843 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.831595 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.831634 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.832834 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.832882 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.832903 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I1007 22:20:04.833184 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.833217 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.834376 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.834409 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.839666 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.839729 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.839753 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.839767 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.840546 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.841417 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.841457 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.842669 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.842711 23944 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1007 22:20:04.842733 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I1007 22:20:04.843002 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.843031 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.843045 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.844107 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.844876 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.844913 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.846115 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.846151 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.851437 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.851487 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.852347 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.852385 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.853586 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.853623 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.858898 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.858949 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:04.859803 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.859841 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.861077 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.861114 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.866373 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.866408 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I1007 22:20:04.866671 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.866702 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.866720 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I1007 22:20:04.866964 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.866994 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.867009 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I1007 22:20:04.867257 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.867289 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.867305 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I1007 22:20:04.867463 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.867492 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.867505 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.868548 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.869382 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.869422 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.870614 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.870662 23944 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1007 22:20:04.870684 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I1007 22:20:04.870860 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.870890 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.870903 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.872086 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.872981 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.873020 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.874202 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.874245 23944 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1007 22:20:04.874266 23944 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I1007 22:20:04.874609 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.874642 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.875849 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.875897 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.877057 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.882336 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.883521 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.888772 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.889974 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.895240 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.895302 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.895328 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.895342 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.896177 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.897099 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.897139 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.898346 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.898390 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.903700 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.903754 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:04.906220 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.906297 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.907529 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.907604 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.912928 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.912978 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.913007 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.913022 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.914156 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.914936 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.914975 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.916217 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.916266 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.921560 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.921625 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.922515 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.922556 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.923784 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.923853 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.929213 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.929268 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:04.930113 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:04.930153 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.931398 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.931449 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.936761 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.936795 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.936823 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.936838 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:04.937949 23944 instancegroups.go:402] Draining the node: "node-2a.local". I1007 22:20:04.938724 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:04.938764 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.939996 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.940045 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.945315 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.945374 23944 instancegroups.go:402] Draining the node: "node-2b.local". I1007 22:20:04.946250 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:04.946291 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.947498 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.947547 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.952854 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.952909 23944 instancegroups.go:402] Draining the node: "node-2c.local". I1007 22:20:04.953743 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:04.953783 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.955022 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.955071 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.960354 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.960391 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I1007 22:20:04.960750 23944 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1007 22:20:04.960790 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.962021 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.962066 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.963248 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.964406 23944 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1007 22:20:04.965693 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.970943 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.971019 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.971048 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.971065 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:04.972046 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:04.972914 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:04.972960 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.974160 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.974197 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.979440 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.979503 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:04.980606 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:04.980651 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.981884 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.981920 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.987245 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.987286 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.987308 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.987322 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:04.988461 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:04.989383 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:04.989423 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.990636 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.990673 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:04.995980 23944 instancegroups.go:505] Cluster validated. I1007 22:20:04.996036 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:04.997012 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:04.997052 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:04.998304 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:04.998340 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.003631 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.003686 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:05.004667 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:05.004708 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.005919 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.005955 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.011213 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.011246 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.011267 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.011282 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:05.012459 23944 instancegroups.go:402] Draining the node: "node-2a.local". I1007 22:20:05.013344 23944 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1007 22:20:05.013382 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.014650 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.014694 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.019967 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.020020 23944 instancegroups.go:402] Draining the node: "node-2b.local". I1007 22:20:05.020909 23944 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1007 22:20:05.020947 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.022151 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.022190 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.027428 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.027482 23944 instancegroups.go:402] Draining the node: "node-2c.local". I1007 22:20:05.028329 23944 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1007 22:20:05.028371 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.029581 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.029622 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.034906 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.034941 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I1007 22:20:05.035640 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.035667 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.035681 23944 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1007 22:20:05.036445 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:05.037252 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:05.037289 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.038489 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.038526 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.043751 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.043811 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:05.045007 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:05.045051 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.046257 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.046291 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.051556 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.051593 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I1007 22:20:05.051917 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.051948 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.051961 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:05.052827 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:05.053686 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:05.053726 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.054929 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.054965 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.060226 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.060279 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:05.061175 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:05.061220 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.062423 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.062458 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.067706 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.067741 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I1007 22:20:05.068066 23944 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled W1007 22:20:05.068121 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1007 22:20:05.068145 23944 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled W1007 22:20:05.068163 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1007 22:20:05.068183 23944 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled W1007 22:20:05.068192 23944 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1007 22:20:05.068212 23944 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1007 22:20:05.068227 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I1007 22:20:05.068446 23944 instancegroups.go:572] Detaching instance "bastion-1a", in group "bastion-1". I1007 22:20:05.068503 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.069718 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.069754 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.075049 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.075110 23944 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled I1007 22:20:05.075182 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.075213 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.075231 23944 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1007 22:20:05.076370 23944 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled I1007 22:20:05.076401 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.076421 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.076433 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:05.077347 23944 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1007 22:20:05.077405 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.078624 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.078661 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.083889 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.083928 23944 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled I1007 22:20:05.083942 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.083962 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.083975 23944 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1007 22:20:05.084921 23944 instancegroups.go:570] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I1007 22:20:05.084978 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.086190 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.086227 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:05.091486 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.091525 23944 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1007 22:20:05.091542 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I1007 22:20:05.091797 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.091831 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.091846 23944 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1007 22:20:05.094180 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:05.094936 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:05.094981 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.096210 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.096261 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.096313 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:05.096376 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:05.097944 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:05.098286 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:05.098339 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.099545 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.099606 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.200066 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.200136 23944 instancegroups.go:402] Draining the node: "node-1e.local". I1007 22:20:05.200226 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:05.201773 23944 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1007 22:20:05.202167 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:05.202229 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.203425 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.203487 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.203539 23944 instancegroups.go:402] Draining the node: "node-1f.local". I1007 22:20:05.204528 23944 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1007 22:20:05.204581 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.205793 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.205857 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.306434 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.306523 23944 instancegroups.go:402] Draining the node: "node-1g.local". I1007 22:20:05.307699 23944 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1007 22:20:05.307758 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.308993 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.309059 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.309080 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I1007 22:20:05.309371 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.309407 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.309422 23944 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1007 22:20:05.311431 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:05.311502 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:05.312769 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:05.313053 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:05.313097 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.314296 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.314375 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.414870 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.414936 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:05.415060 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:05.416871 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:05.416955 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:05.417005 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.418213 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.418264 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.418310 23944 instancegroups.go:402] Draining the node: "node-1e.local". I1007 22:20:05.419788 23944 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1007 22:20:05.419836 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.420984 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.421076 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.521573 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.521635 23944 instancegroups.go:402] Draining the node: "node-1f.local". I1007 22:20:05.522591 23944 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1007 22:20:05.522636 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.523867 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.523912 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.523930 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I1007 22:20:05.524167 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.524199 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.524214 23944 instancegroups.go:313] Tainting 7 nodes in "master-1" instancegroup. I1007 22:20:05.526401 23944 instancegroups.go:402] Draining the node: "master-1a.local". I1007 22:20:05.527198 23944 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1007 22:20:05.527242 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.528452 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.528495 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.528537 23944 instancegroups.go:402] Draining the node: "master-1c.local". I1007 22:20:05.528602 23944 instancegroups.go:402] Draining the node: "master-1b.local". I1007 22:20:05.530187 23944 instancegroups.go:593] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I1007 22:20:05.530600 23944 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1007 22:20:05.530666 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.531847 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.531918 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.632547 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.632600 23944 instancegroups.go:402] Draining the node: "master-1e.local". I1007 22:20:05.632684 23944 instancegroups.go:402] Draining the node: "master-1d.local". I1007 22:20:05.634100 23944 instancegroups.go:593] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I1007 22:20:05.634451 23944 instancegroups.go:593] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I1007 22:20:05.634493 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.635722 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.635765 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.635805 23944 instancegroups.go:402] Draining the node: "master-1f.local". I1007 22:20:05.636732 23944 instancegroups.go:593] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I1007 22:20:05.636789 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.638008 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.638074 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.738740 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.738790 23944 instancegroups.go:402] Draining the node: "master-1g.local". I1007 22:20:05.739675 23944 instancegroups.go:593] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I1007 22:20:05.739719 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.740926 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.740982 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.741012 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I1007 22:20:05.741369 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.741412 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.741428 23944 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1007 22:20:05.743329 23944 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1007 22:20:05.743403 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.744646 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.744695 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.744710 23944 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1007 22:20:05.744770 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.745984 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.746031 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.746081 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:05.746154 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:05.747482 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:05.747812 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:05.747855 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.749060 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.749127 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.849737 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.849788 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:05.849876 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:05.851129 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:05.851453 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:05.851505 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.852729 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.852772 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.852814 23944 instancegroups.go:402] Draining the node: "node-1e.local". I1007 22:20:05.853654 23944 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1007 22:20:05.853704 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.854914 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.854985 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.955482 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.955534 23944 instancegroups.go:402] Draining the node: "node-1f.local". I1007 22:20:05.956376 23944 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1007 22:20:05.956425 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.957621 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.957668 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.957686 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I1007 22:20:05.957917 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.957956 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.957970 23944 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1007 22:20:05.959884 23944 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1007 22:20:05.959929 23944 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1007 22:20:05.959975 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:05.961185 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.961229 23944 instancegroups.go:505] Cluster validated. I1007 22:20:05.961275 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:05.961342 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:05.962632 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:05.962919 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:05.962967 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:05.964171 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:05.964254 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.064844 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.064895 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:06.064933 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:06.066271 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:06.066560 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:06.066603 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.067827 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.067870 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.067911 23944 instancegroups.go:402] Draining the node: "node-1e.local". I1007 22:20:06.068756 23944 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1007 22:20:06.068805 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.070013 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.070073 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.170891 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.170950 23944 instancegroups.go:402] Draining the node: "node-1f.local". I1007 22:20:06.171789 23944 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1007 22:20:06.171841 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.173049 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.173092 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.173109 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I1007 22:20:06.173353 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.173383 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.173396 23944 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1007 22:20:06.174126 23944 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1007 22:20:06.174159 23944 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I1007 22:20:06.174196 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:06.175407 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.175442 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.180690 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.180743 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:06.180767 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:06.182080 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:06.182123 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.182391 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:06.182426 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.183345 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.183380 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.188910 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.188943 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I1007 22:20:06.189188 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.189217 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.189252 23944 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1007 22:20:06.189957 23944 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E1007 22:20:06.190016 23944 instancegroups.go:162] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I1007 22:20:06.190029 23944 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E1007 22:20:06.190068 23944 instancegroups.go:162] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I1007 22:20:06.190099 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:06.191307 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.191352 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.196627 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.196681 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:06.196720 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:06.198090 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:06.198131 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.198405 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:06.198438 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.199348 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.199383 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.205013 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.205046 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I1007 22:20:06.206297 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.206364 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.206390 23944 instancegroups.go:313] Tainting 4 nodes in "node-1" instancegroup. I1007 22:20:06.209353 23944 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1007 22:20:06.209450 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:06.210675 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.210731 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.210746 23944 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1007 22:20:06.210807 23944 instancegroups.go:174] waiting for 1ms after detaching instance I1007 22:20:06.212019 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.212075 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.212133 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:06.212226 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:06.212284 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:06.214131 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:06.214188 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.214543 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:06.214596 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.214922 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:06.214974 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.215378 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.316192 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.316244 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:06.317076 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:06.317125 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.318353 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.318399 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.318417 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I1007 22:20:06.318651 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.318688 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.318703 23944 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1007 22:20:06.320950 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:06.321738 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:06.321784 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.323010 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.323059 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.323105 23944 instancegroups.go:402] Draining the node: "node-1e.local". I1007 22:20:06.323178 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:06.324512 23944 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1007 22:20:06.324800 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:06.324844 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.326064 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.326134 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.426962 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.427014 23944 instancegroups.go:402] Draining the node: "node-1g.local". I1007 22:20:06.427048 23944 instancegroups.go:402] Draining the node: "node-1f.local". I1007 22:20:06.428406 23944 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1007 22:20:06.428700 23944 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1007 22:20:06.428744 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.429960 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.430004 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.430043 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:06.430915 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:06.430960 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.432175 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.432242 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.533078 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.533131 23944 instancegroups.go:402] Draining the node: "node-1d.local". I1007 22:20:06.534005 23944 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1007 22:20:06.534052 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.535280 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.535323 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.535341 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I1007 22:20:06.535561 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.535588 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.535603 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1007 22:20:06.535622 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1007 22:20:06.535637 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1007 22:20:06.535655 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I1007 22:20:06.535799 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.535823 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.535836 23944 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1007 22:20:06.536831 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1007 22:20:06.536862 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1007 22:20:06.536878 23944 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1007 22:20:06.536925 23944 instancegroups.go:402] Draining the node: "node-1a.local". I1007 22:20:06.537734 23944 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1007 22:20:06.537770 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.539006 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.539043 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.544310 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.544362 23944 instancegroups.go:402] Draining the node: "node-1b.local". I1007 22:20:06.545193 23944 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1007 22:20:06.545231 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.546455 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.546492 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.551741 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.551792 23944 instancegroups.go:402] Draining the node: "node-1c.local". I1007 22:20:06.552636 23944 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1007 22:20:06.552673 23944 instancegroups.go:438] waiting for 1ms after terminating instance I1007 22:20:06.553919 23944 instancegroups.go:472] Validating the cluster. I1007 22:20:06.553957 23944 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1007 22:20:06.559241 23944 instancegroups.go:505] Cluster validated. I1007 22:20:06.559274 23944 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.989s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.023s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) PASS ok k8s.io/kops/pkg/k8scodecs 0.056s === RUN TestParse E1007 22:19:39.096869 19718 versions.go:96] unable to parse Kubernetes version "" E1007 22:19:39.096943 19718 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.050s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.103s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.20s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.06s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.410s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.090s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.080s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.077s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I1007 22:19:47.157146 21900 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.135s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I1007 22:19:50.623456 23107 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W1007 22:19:50.681756 23107 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W1007 22:19:50.683615 23107 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.154s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I1007 22:19:49.999237 22920 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.121s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.146s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.086s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.01s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.197s === RUN TestServerGroupBuilder I1007 22:19:51.762691 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.776463 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.790093 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.806073 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.881577 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.912363 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.929680 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.961688 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.975937 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.065872 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.071533 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.077085 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.082165 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.089863 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:52.096814 23388 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.34s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.404s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.082s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.011s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.060s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.02s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.02s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.20s) --- PASS: TestIssueCert/ca (0.49s) --- PASS: TestIssueCert/client (0.32s) --- PASS: TestIssueCert/clientOneYear (0.39s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.257s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.029s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I1007 22:19:51.639622 23360 routetable.go:53] DescribeRouteTables: { } W1007 22:19:51.639753 23360 tags.go:67] cluster tag not found on route-table:rtb-1234 I1007 22:19:51.639768 23360 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I1007 22:19:51.639883 23360 iaminstanceprofile.go:193] ListInstanceProfiles: { } W1007 22:19:51.639945 23360 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I1007 22:19:51.640037 23360 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I1007 22:19:51.640149 23360 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I1007 22:19:51.640248 23360 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I1007 22:19:51.640340 23360 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I1007 22:19:51.640432 23360 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I1007 22:19:51.640476 23360 volumes.go:128] DescribeVolumes: { } W1007 22:19:51.640521 23360 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I1007 22:19:51.640660 23360 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:51.640848 23360 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:51.640886 23360 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:51.640971 23360 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:51.641037 23360 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:51.641097 23360 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:51.641118 23360 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:51.641142 23360 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:51.641170 23360 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:51.641194 23360 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I1007 22:19:51.641229 23360 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I1007 22:19:51.641255 23360 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I1007 22:19:51.641308 23360 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I1007 22:19:51.641334 23360 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I1007 22:19:51.641365 23360 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I1007 22:19:51.641403 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I1007 22:19:51.641430 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I1007 22:19:51.641462 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I1007 22:19:51.641489 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I1007 22:19:51.641517 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I1007 22:19:51.641551 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I1007 22:19:51.641580 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I1007 22:19:51.641603 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I1007 22:19:51.641632 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I1007 22:19:51.641656 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I1007 22:19:51.641683 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I1007 22:19:51.641704 23360 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.067s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.042s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.082s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.019s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.016s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.052s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.017s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.004s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.01s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.01s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.111s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.065s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.038s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W1007 22:19:43.866873 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.866931 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.866941 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.866950 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.866959 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.866967 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.866976 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.866985 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.866993 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.867003 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.867011 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.867020 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.867028 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.867037 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.867046 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.867055 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.869619 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.869664 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.869732 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.869745 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.869801 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.869813 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.869869 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.869881 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.869937 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.869949 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870010 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870022 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870079 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870091 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870164 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870176 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870245 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870257 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870312 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870324 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870379 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870391 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870464 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870477 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870542 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870556 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870611 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870623 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870680 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870691 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870752 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870764 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870824 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870836 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870891 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870902 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.870956 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.870976 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871078 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871091 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871145 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871157 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871215 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871229 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871284 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871296 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871354 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871366 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871420 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871432 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871486 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871499 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871579 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871592 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871652 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871665 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871732 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871745 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871798 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871810 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871863 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871877 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.871935 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.871949 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872015 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872027 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872083 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872095 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872148 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872160 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872214 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872225 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872292 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872305 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872364 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872376 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872432 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872444 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872499 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872512 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872595 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872607 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872661 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872673 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872748 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872760 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872841 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872854 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872910 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872921 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.872977 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.872989 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873052 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873065 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873124 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873136 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873190 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873203 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873264 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873277 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873330 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873342 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873412 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873426 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873483 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873496 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873553 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873567 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873625 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873637 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873696 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873708 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873761 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873773 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873835 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873850 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873909 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873921 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.873975 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.873987 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874040 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874052 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874107 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874119 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874201 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874216 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874275 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874287 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874346 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874358 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874412 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874424 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874482 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874494 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874548 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874560 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874629 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874642 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874705 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874719 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874776 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874788 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874844 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874856 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874916 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.874928 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.874986 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875011 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875069 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875081 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875140 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875151 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875206 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875218 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875277 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875290 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875347 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875359 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875419 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875431 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875486 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875498 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875554 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875566 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875620 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875632 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875718 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875730 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875792 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875805 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875872 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875886 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.875943 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.875956 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876016 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876028 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876083 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876095 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876149 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876161 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876216 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876228 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876283 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876295 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876353 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876366 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876424 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876439 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876495 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876507 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876566 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876578 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876636 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876649 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876705 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876717 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.876770 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.876782 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W1007 22:19:43.879484 21243 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1007 22:19:43.879499 21243 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1007 22:19:43.879509 21243 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1007 22:19:43.879517 21243 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1007 22:19:43.879526 21243 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1007 22:19:43.879537 21243 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1007 22:19:43.879545 21243 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1007 22:19:43.879574 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879587 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879596 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.879605 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879615 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.879624 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879633 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.879642 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879651 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.879660 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879669 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879681 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879691 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879700 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.879709 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.879719 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888210 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888248 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888324 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888344 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888410 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888428 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888498 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888516 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888634 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888653 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888714 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888730 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888789 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888805 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888867 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888885 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.888944 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.888960 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889018 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889035 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889099 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889116 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889172 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889193 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889276 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889295 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889357 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889373 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889432 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889446 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889504 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889521 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889581 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889600 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889668 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889686 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889755 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889773 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889839 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889858 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889917 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889929 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.889985 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.889998 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890055 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890068 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890125 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890138 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890207 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890220 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890283 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890296 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890351 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890364 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890423 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890436 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890549 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890569 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890631 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890644 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890702 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890714 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890772 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890785 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890843 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890856 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890914 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890927 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.890982 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.890995 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.891632 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.891649 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.891716 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.891730 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.891812 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.891828 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.891892 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.891905 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.891963 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.891976 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892031 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892043 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892098 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892119 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892186 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892199 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892394 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892433 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892516 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892553 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892633 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892668 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892751 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892788 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892871 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.892907 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.892988 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893023 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893101 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893150 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893236 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893275 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893363 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893399 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893479 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893515 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893644 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893682 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893769 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893807 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.893898 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.893936 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894019 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894055 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894135 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894171 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894256 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894293 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894383 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894421 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894505 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894544 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894626 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894663 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894744 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894781 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894865 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.894902 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.894985 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895021 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895104 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895141 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895226 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895262 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895345 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895380 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895463 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895500 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895579 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895616 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895694 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895729 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895810 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895846 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.895927 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.895963 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896045 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896081 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896165 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896200 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896283 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896319 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896401 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896437 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896561 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896598 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896680 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896716 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896795 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896834 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.896915 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.896951 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897030 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897066 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897164 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897204 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897289 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897333 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897419 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897454 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897532 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897716 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897797 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897832 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.897909 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.897942 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898020 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898054 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898135 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898172 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898249 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898283 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898361 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898397 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898475 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898512 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898587 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898621 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898694 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898727 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898803 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898838 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.898911 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.898994 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.899073 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.899107 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1007 22:19:43.899182 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1007 22:19:43.899215 21243 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.073s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.048s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.02s) PASS ok k8s.io/kops/tests/codecs 0.073s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.04s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.175s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.12s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.03s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.06s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.03s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.02s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.288s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I1007 22:19:47.790592 22044 files.go:57] Writing file "/tmp/TestWriteFile524477063/001/SubDir/test1.tmp" I1007 22:19:47.790780 22044 files.go:113] Changing file mode for "/tmp/TestWriteFile524477063/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.090s === RUN TestBootstrapChannelBuilder_BuildTasks I1007 22:19:56.546757 23791 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:56.546955 23791 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:56.546996 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:56.547028 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:56.547069 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:56.547110 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:56.547124 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:56.547176 23791 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:56.547216 23791 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:56.555226 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.608853 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.713225 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.782115 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.847549 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.928289 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:56.995465 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:57.096267 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:57.218649 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:57.725516 23791 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.24s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I1007 22:19:57.791617 23791 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:57.791873 23791 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:57.791984 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:57.792036 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:57.792087 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:57.792132 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:57.792161 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:57.792199 23791 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:57.792229 23791 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:57.792260 23791 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I1007 22:19:57.794196 23791 zones.go:93] ListHostedZonesPages { } I1007 22:19:57.876786 23791 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.09s) === RUN TestBootstrapChannelBuilder_AWSCloudController I1007 22:19:57.877030 23791 featureflag.go:160] FeatureFlag "ImageDigest"=false I1007 22:19:57.877263 23791 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:57.877334 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1007 22:19:57.877381 23791 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1007 22:19:57.877425 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1007 22:19:57.877463 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1007 22:19:57.877490 23791 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1007 22:19:57.877523 23791 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1007 22:19:57.877547 23791 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1007 22:19:57.879350 23791 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.12s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.07s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.13s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W1007 22:19:58.221940 23791 new_cluster.go:1087] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I1007 22:19:58.264645 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.264685 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.264705 23791 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1007 22:19:58.264739 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1007 22:19:58.264758 23791 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I1007 22:19:58.264799 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1007 22:19:58.264834 23791 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I1007 22:19:58.264866 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.264887 23791 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1007 22:19:58.264917 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.264937 23791 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1007 22:19:58.264969 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.265020 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.265054 23791 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1007 22:19:58.265072 23791 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1007 22:19:58.265090 23791 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I1007 22:19:58.265107 23791 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I1007 22:19:58.265125 23791 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I1007 22:19:58.265145 23791 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1007 22:19:58.265164 23791 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I1007 22:19:58.265184 23791 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I1007 22:19:58.266957 23791 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.806s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I1007 22:19:53.951155 23686 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender23114619/001 I1007 22:19:53.953063 23686 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender23114619/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I1007 22:19:53.954318 23686 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender2806307669/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I1007 22:19:53.955083 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1007 22:19:53.955176 23686 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1007 22:19:53.955254 23686 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I1007 22:19:53.955472 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.955576 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.955650 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.955675 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.955838 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.955880 23686 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1007 22:19:53.955923 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.955999 23686 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1007 22:19:53.956099 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1007 22:19:53.956235 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.956298 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.956349 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.956370 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.956409 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.956468 23686 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1007 22:19:53.956539 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I1007 22:19:53.956842 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.956901 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.957014 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.957061 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.957095 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.957128 23686 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1007 22:19:53.957303 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.957509 23686 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1007 22:19:53.957559 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.957588 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.957619 23686 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1007 22:19:53.957680 23686 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.957890 23686 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I1007 22:19:53.957957 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.957984 23686 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I1007 22:19:53.958027 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1007 22:19:53.958173 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.958224 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.958277 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.958296 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.958326 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.958357 23686 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1007 22:19:53.958406 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.958470 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.958501 23686 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1007 22:19:53.958545 23686 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.958601 23686 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I1007 22:19:53.958630 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1007 22:19:53.958678 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I1007 22:19:53.958841 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1007 22:19:53.958890 23686 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1007 22:19:53.958919 23686 internetgateways.go:66] CreateInternetGateway: { } I1007 22:19:53.958948 23686 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I1007 22:19:53.959072 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.959120 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.959167 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.959185 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.959260 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.959291 23686 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1007 22:19:53.959329 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.959407 23686 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1007 22:19:53.959488 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1007 22:19:53.959619 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.959666 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.959715 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.959735 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.959772 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.959815 23686 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1007 22:19:53.959882 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I1007 22:19:53.960779 23686 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender698291751/001 I1007 22:19:53.960997 23686 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender698291751/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I1007 22:19:53.963564 23686 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3881932320/001 I1007 22:19:53.965177 23686 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3881932320/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I1007 22:19:53.965742 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.965798 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.965913 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.965958 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.965984 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.966016 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.966084 23686 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1007 22:19:53.966298 23686 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I1007 22:19:53.966346 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1007 22:19:53.966381 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1007 22:19:53.966515 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.966563 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.966618 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.966638 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.966667 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.966694 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.966739 23686 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1007 22:19:53.966827 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I1007 22:19:53.967282 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.967329 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.967451 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.967491 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.967516 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.967544 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.967599 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.967718 23686 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1007 22:19:53.967763 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.967788 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.967814 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1007 22:19:53.967943 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.968001 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.968052 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.968078 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.968106 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.968132 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.968175 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.968233 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.968263 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I1007 22:19:53.968565 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.968625 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.968717 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.968759 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.968784 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.968809 23686 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1007 22:19:53.968856 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.968961 23686 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1007 22:19:53.969015 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.969047 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.969076 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.969799 23686 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:19:53.970015 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1007 22:19:53.970658 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.970742 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.970830 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.970847 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.970878 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.970904 23686 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1007 22:19:53.970945 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.971009 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.971031 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.971081 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I1007 22:19:53.971424 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.971462 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.971538 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.971576 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.971596 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.971616 23686 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1007 22:19:53.971645 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.971710 23686 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1007 22:19:53.971746 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.971922 23686 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1007 22:19:53.971976 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.971998 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.972019 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1007 22:19:53.972150 23686 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1007 22:19:53.972186 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.972227 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.972242 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.972262 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.972282 23686 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1007 22:19:53.972334 23686 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1007 22:19:53.972397 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.972482 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.972506 23686 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I1007 22:19:53.972658 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I1007 22:19:53.972729 23686 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I1007 22:19:53.972838 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.972874 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.972905 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.972917 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.972973 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.973000 23686 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1007 22:19:53.973027 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.973056 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.973131 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1007 22:19:53.973151 23686 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1007 22:19:53.973178 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1007 22:19:53.973295 23686 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1007 22:19:53.973329 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.973358 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.973370 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.973395 23686 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1007 22:19:53.973421 23686 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1007 22:19:53.973468 23686 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I1007 22:19:53.973651 23686 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:53.973696 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.973778 23686 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1007 22:19:53.973833 23686 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1007 22:19:53.973853 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.973888 23686 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:53.973964 23686 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:53.973997 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1007 22:19:53.974030 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.974050 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.974069 23686 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1007 22:19:53.974088 23686 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I1007 22:19:53.974224 23686 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I1007 22:19:53.974308 23686 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:53.974341 23686 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I1007 22:19:53.974374 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1007 22:19:53.974386 23686 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1007 22:19:53.974409 23686 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.073s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.048s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.085s === RUN TestDiskRenderAzure I1007 22:19:49.180121 22523 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I1007 22:19:49.180415 22523 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I1007 22:19:49.180799 22523 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I1007 22:19:49.181332 22523 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I1007 22:19:49.181578 22523 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I1007 22:19:49.181828 22523 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I1007 22:19:49.182075 22523 resourcegroup.go:103] Creating a new Resource Group with name: rg I1007 22:19:49.182089 22523 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I1007 22:19:49.182286 22523 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I1007 22:19:49.182812 22523 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I1007 22:19:49.183037 22523 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I1007 22:19:49.183291 22523 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I1007 22:19:49.183562 22523 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I1007 22:19:49.183791 22523 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.059s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.075s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I1007 22:19:52.975205 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.975682 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.975802 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.976181 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.976285 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.976421 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I1007 22:19:52.976586 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.976673 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.976778 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.976892 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.977012 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.977066 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I1007 22:19:52.977241 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.977444 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.977541 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.977811 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1007 22:19:52.977908 23555 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1007 22:19:52.978030 23555 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.047s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.046s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.061s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.048s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.059s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.034s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I1007 22:19:56.125634 23752 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I1007 22:19:56.125702 23752 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I1007 22:19:56.125716 23752 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I1007 22:19:56.125732 23752 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I1007 22:19:56.125746 23752 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I1007 22:19:56.125760 23752 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I1007 22:19:56.125774 23752 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W1007 22:19:56.126279 23752 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.050s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.072s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.057s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.078s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.035s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.018s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W1007 22:19:52.238700 23494 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.024s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.007s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.038s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.011s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.019s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W1007 22:19:56.530476 23790 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.053s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 128.4 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.0-r1.apk >>> kops: Build complete at Fri, 07 Oct 2022 22:20:59 +0000 elapsed time 0h 3m 27s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20221007.221733) (2/3) Purging kubernetes (1.25.0-r2) (3/3) Purging go (1.19.2-r0) Executing busybox-1.35.0-r27.trigger OK: 393 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...