>>> kops: Building testing/kops 1.25.3-r5 (using abuild 3.11.0_rc13-r0) started Wed, 03 May 2023 19:22:30 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.26.1-r4) (2/3) Installing go (1.20.4-r0) (3/3) Installing .makedepends-kops (20230503.192232) Executing busybox-1.36.0-r8.trigger OK: 563 MiB in 104 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 90 31.1M 90 28.2M 0 0 27.6M 0 0:00:01 0:00:01 --:--:-- 27.6M 100 31.1M 100 31.1M 0 0 28.9M 0 0:00:01 0:00:01 --:--:-- 28.9M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading google.golang.org/api v0.103.0 go: downloading k8s.io/api v0.25.4 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/spf13/viper v1.14.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/apimachinery v0.25.4 go: downloading k8s.io/client-go v0.25.4 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading k8s.io/klog v1.0.0 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading github.com/stretchr/testify v1.8.1 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading golang.org/x/crypto v0.3.0 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.2.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/google/go-cmp v0.5.9 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading golang.org/x/net v0.2.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/kubectl v0.25.4 go: downloading k8s.io/component-base v0.25.4 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading golang.org/x/sys v0.2.0 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.4 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.1.0 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.4.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading google.golang.org/grpc v1.50.1 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading go.opencensus.io v0.24.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/kr/text v0.2.0 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0503 19:24:45.982055 50471 addon.go:233] addon test wants to update all nodes I0503 19:24:45.983792 50471 addon.go:233] addon test wants to update worker nodes I0503 19:24:45.984122 50471 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0503 19:24:45.984515 50471 addon.go:266] installing PKI for "test" I0503 19:24:47.138009 50471 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.44s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.525s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.073s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestCreateClusterMinimal I0503 19:24:47.970696 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:47.970937 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:47.971012 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:47.971086 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:47.971169 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:47.971236 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:47.971267 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:47.971318 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:47.971364 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:47.971489 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:47.971545 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.041459 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.042181 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.042698 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.054915 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.055001 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.055036 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.055068 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.055111 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.055140 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.055156 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.055179 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.055195 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.055257 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.055282 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.106051 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.106285 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.106668 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.112366 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.112461 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.112518 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.112556 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.112605 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.112648 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.112683 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.112725 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.112750 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.112833 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.112865 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.153500 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.153778 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.154226 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.162603 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.162693 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.162727 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.162759 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.162791 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.162817 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.162835 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.162860 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.162876 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.162936 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.162960 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.227494 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.227760 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.228205 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.238075 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.238188 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.238236 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.238286 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.238343 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.238705 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.238740 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.238780 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.238807 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.238907 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.238964 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.342211 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.342390 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.342670 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.350063 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.350163 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.350200 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.350233 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.350267 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.350292 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.350309 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.350331 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.350348 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.350414 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.350439 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.380957 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.381120 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.381426 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.386862 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.386944 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.386980 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.387010 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.387042 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.387068 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.387086 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.387108 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.387125 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.387218 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.387247 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.424637 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.424795 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.425077 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.430459 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.430532 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.430561 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.430587 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.430616 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.430641 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.430658 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.430680 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.430696 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.430759 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.430784 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.472628 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.472719 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.473002 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.51s) === RUN TestCreateClusterHetzner I0503 19:24:48.478429 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.478529 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.478565 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.478593 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.478623 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.478653 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.478671 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.478692 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.478708 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.478776 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.478801 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0503 19:24:48.525800 51914 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0503 19:24:48.525850 51914 new_cluster.go:1279] Cloud Provider ID = hetzner I0503 19:24:48.533034 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.533114 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.533146 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.533176 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.533208 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.533234 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.533253 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.533276 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.533294 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.533353 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.533379 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.576913 51914 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.10s) === RUN TestCreateClusterOverride I0503 19:24:48.582287 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.582398 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.582442 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.582486 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.582536 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.582571 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.582607 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.582649 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.582673 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.582762 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.582794 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.620615 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.621120 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.621402 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.04s) === RUN TestCreateClusterComplex I0503 19:24:48.626777 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.626879 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.626920 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.626961 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.627015 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.627055 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.627087 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.627127 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.627152 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.627241 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.627274 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.674713 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.674880 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.675182 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.05s) === RUN TestCreateClusterHA I0503 19:24:48.680506 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.680593 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.680625 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.680655 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.680685 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.680710 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.680727 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.680749 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.680768 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.680838 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.680863 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.706050 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.706363 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.706378 51914 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0503 19:24:48.706387 51914 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0503 19:24:48.706696 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:48.714783 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.714864 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.714900 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.714929 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.714959 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.714984 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.715001 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.715023 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.715039 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.715096 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.715120 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.791609 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:48.791960 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:48.791977 51914 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0503 19:24:48.791987 51914 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0503 19:24:48.792310 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.13s) === RUN TestCreateClusterHAGCE I0503 19:24:48.806182 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.806324 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.806373 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.806426 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.806481 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.806521 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.806553 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.806589 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.806617 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.806723 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.806765 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.895579 51914 new_cluster.go:1279] Cloud Provider ID = gce I0503 19:24:48.895726 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.10s) === RUN TestCreateClusterGCE I0503 19:24:48.904224 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.904317 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.904352 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.904380 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.904411 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.904439 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.904456 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.904478 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.904495 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.904565 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.904591 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.960633 51914 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0503 19:24:48.960709 51914 new_cluster.go:1279] Cloud Provider ID = gce I0503 19:24:48.960778 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.06s) === RUN TestCreateClusterHASharedZone I0503 19:24:48.966351 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:48.966449 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:48.966481 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:48.966513 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:48.966545 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:48.966571 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:48.966589 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:48.966612 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:48.966630 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:48.966700 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:48.966726 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0503 19:24:49.027029 51914 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0503 19:24:49.027085 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.027312 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.027620 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.07s) === RUN TestCreateClusterHASharedZones I0503 19:24:49.034398 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.034488 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.034523 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.034552 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.034582 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.034607 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.034628 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.034651 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.034668 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.034740 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.034766 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0503 19:24:49.094729 51914 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0503 19:24:49.094791 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.095128 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.095143 51914 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0503 19:24:49.095470 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.07s) === RUN TestCreateClusterPrivate I0503 19:24:49.106451 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.106544 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.106577 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.106606 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.106638 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.106663 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.106679 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.106699 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.106714 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.106784 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.106809 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.148272 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.148631 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.148658 51914 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0503 19:24:49.149180 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:49.149223 51914 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I0503 19:24:49.160966 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.161117 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.161166 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.161215 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.161270 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.161313 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.161347 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.161389 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.161416 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.161523 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.161566 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.224707 51914 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0503 19:24:49.224779 51914 new_cluster.go:1279] Cloud Provider ID = gce I0503 19:24:49.224900 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.07s) === RUN TestCreateClusterWithNGWSpecified I0503 19:24:49.231556 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.231650 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.231686 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.231712 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.231742 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.231767 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.231782 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.231803 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.231822 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.231895 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.231921 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.267586 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.267801 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.267818 51914 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0503 19:24:49.268123 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:49.268154 51914 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.04s) === RUN TestCreateClusterWithINGWSpecified I0503 19:24:49.274291 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.274390 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.274478 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.274517 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.274552 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.274586 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.274607 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.274630 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.274647 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.274731 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.274767 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.313269 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.313470 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.313486 51914 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0503 19:24:49.313789 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:49.313819 51914 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.05s) === RUN TestCreateClusterSharedVPC I0503 19:24:49.319861 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.319943 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.319976 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.320003 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.320034 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.320059 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.320075 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.320096 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.320113 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.320185 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.320210 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.353144 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.353298 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.353352 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.353404 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.353430 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.353473 51914 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0503 19:24:49.353762 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.04s) === RUN TestCreateClusterSharedSubnets I0503 19:24:49.358938 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.359047 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.359083 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.359112 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.359145 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.359169 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.359187 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.359209 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.359226 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.359296 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.359322 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.391603 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.391667 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.391775 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.392006 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.392040 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.392082 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.392109 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.392590 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.04s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0503 19:24:49.400839 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.400959 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.401005 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.401049 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.401094 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.401131 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.401158 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.401192 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.401215 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.401318 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.401358 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.475461 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:24:49.475545 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.475570 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.475651 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.475788 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.475810 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.475834 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.475848 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.476149 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.08s) === RUN TestCreateClusterPrivateSharedSubnets I0503 19:24:49.482207 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.482317 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.482351 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.482386 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.482418 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.482445 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.482463 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.482486 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.482502 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.482630 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.482658 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.515059 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.515105 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.515174 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.515204 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.515228 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.515361 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.515381 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.515404 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:24:49.515417 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0503 19:24:49.515774 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.04s) === RUN TestCreateClusterIPv6 I0503 19:24:49.521169 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.521254 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.521284 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.521309 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.521338 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.521363 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.521379 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.521400 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.521419 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.521486 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.521510 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.553709 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.553917 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.554404 51914 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.04s) === RUN TestCreateClusterDifferentAMIs I0503 19:24:49.561099 51914 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0503 19:24:49.561122 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.561237 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.561278 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.561325 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.561371 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.561411 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.561453 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.561491 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.561527 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.561637 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.561678 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.617370 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.617451 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.617467 51914 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0503 19:24:49.617774 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:49.624713 51914 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.06s) === RUN TestCreateClusterKarpenter I0503 19:24:49.624806 51914 featureflag.go:160] FeatureFlag "Karpenter"=true I0503 19:24:49.624825 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.624936 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.624980 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.625018 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.625059 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.625097 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.625126 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.625162 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.625186 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.625282 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.625323 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0503 19:24:49.681005 51914 new_cluster.go:1279] Cloud Provider ID = aws I0503 19:24:49.681289 51914 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0503 19:24:49.681313 51914 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0503 19:24:49.681329 51914 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0503 19:24:49.681865 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:49.690921 51914 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.07s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0503 19:24:49.691261 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.691349 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.691384 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.691448 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.691485 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.691521 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.691541 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.691570 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.691589 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.698125 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0503 19:24:49.699526 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:49.699611 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:49.699638 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:49.699665 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:49.699692 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:49.699765 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:49.699782 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:49.699802 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:49.699843 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:49.713977 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:24:49.715696 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:49.715753 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0503 19:24:52.782238 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:55.265630 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:24:55.266145 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:24:55.268391 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:24:55.268415 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:24:55.268544 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:24:55.268561 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:24:55.268640 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:24:55.268713 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:24:55.274646 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:24:55.275743 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:55.275978 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:55.276618 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:55.277064 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:55.277134 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:24:55.277402 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:24:55.277680 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:24:55.307715 51914 target.go:112] Terraform output is in /tmp/test2084851154/out --- PASS: TestMinimal (5.61s) === RUN TestMinimal_v1_23 I0503 19:24:55.309824 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:55.309925 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:55.309972 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:55.310018 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:55.310069 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:55.310138 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:55.310172 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:55.310210 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:55.310237 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:55.325429 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0503 19:24:55.327314 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:55.327389 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.655832 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:56.752946 51914 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0503 19:24:56.753167 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:24:56.753181 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:24:56.753241 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:24:56.753329 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:24:56.753379 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:24:56.753476 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:24:56.758382 51914 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0503 19:24:56.763005 51914 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0503 19:24:56.763200 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.763320 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.764043 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.765515 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.765618 51914 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0503 19:24:56.765837 51914 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0503 19:24:56.765960 51914 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0503 19:24:56.801194 51914 target.go:112] Terraform output is in /tmp/test235130458/out --- PASS: TestMinimal_v1_23 (1.49s) === RUN TestMinimal_v1_24 I0503 19:24:56.803433 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:56.803523 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:56.803563 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:56.803592 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:56.803624 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:56.803681 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:56.803698 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:56.803720 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:56.803737 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:56.819167 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:24:56.821606 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:56.821691 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:57.907082 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:58.019489 51914 executor.go:111] Tasks: 0 done / 93 total; 46 can run W0503 19:24:58.020450 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:24:58.020952 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:24:58.021024 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:24:58.023661 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:24:58.023687 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:24:58.023792 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:24:58.025447 51914 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0503 19:24:58.031015 51914 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0503 19:24:58.031485 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:58.031812 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:58.034609 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:58.036406 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:58.037444 51914 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0503 19:24:58.037795 51914 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0503 19:24:58.037985 51914 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0503 19:24:58.086943 51914 target.go:112] Terraform output is in /tmp/test1280745638/out --- PASS: TestMinimal_v1_24 (1.29s) === RUN TestMinimal_v1_25 I0503 19:24:58.093082 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:58.093258 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:58.093322 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:58.093376 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:58.093428 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:58.093523 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:58.093553 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:58.093590 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:58.093618 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:58.121089 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:24:58.124355 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:58.124453 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.462407 51914 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.618692 51914 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0503 19:24:59.621314 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:24:59.621764 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:24:59.621885 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:24:59.622562 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:24:59.622583 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:24:59.622636 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:24:59.623813 51914 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0503 19:24:59.629640 51914 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0503 19:24:59.630254 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.631528 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.631599 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.634391 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.634558 51914 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0503 19:24:59.634838 51914 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0503 19:24:59.635004 51914 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0503 19:24:59.694172 51914 target.go:112] Terraform output is in /tmp/test2907653039/out --- PASS: TestMinimal_v1_25 (1.60s) === RUN TestMinimal_v1_26 I0503 19:24:59.697459 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.697586 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.697633 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.697682 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.697729 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.697812 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.697841 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.697880 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.697904 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.725117 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:24:59.728881 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:24:59.729004 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:01.081648 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:01.226798 51914 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0503 19:25:01.230545 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:01.230638 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:01.230582 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:01.230746 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:01.231447 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:01.231460 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:01.233938 51914 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0503 19:25:01.239432 51914 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0503 19:25:01.240025 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:01.240314 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:01.241363 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:01.242675 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:01.242785 51914 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0503 19:25:01.243202 51914 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0503 19:25:01.243397 51914 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0503 19:25:01.280601 51914 target.go:112] Terraform output is in /tmp/test2029187844/out --- PASS: TestMinimal_v1_26 (1.59s) === RUN TestHetzner I0503 19:25:01.283049 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:02.487096 51914 executor.go:111] Tasks: 0 done / 50 total; 39 can run I0503 19:25:02.489977 51914 executor.go:111] Tasks: 39 done / 50 total; 5 can run I0503 19:25:02.491921 51914 executor.go:111] Tasks: 44 done / 50 total; 2 can run I0503 19:25:02.497305 51914 executor.go:111] Tasks: 46 done / 50 total; 4 can run I0503 19:25:02.499958 51914 executor.go:111] Tasks: 50 done / 50 total; 0 can run I0503 19:25:02.515181 51914 target.go:112] Terraform output is in /tmp/test3794284774/out --- PASS: TestHetzner (1.23s) === RUN TestNvidia I0503 19:25:02.518067 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:02.518189 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:02.518237 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:02.518283 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:02.518329 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:02.518413 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:02.518442 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:02.518480 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:02.518507 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:02.543247 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:02.545061 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.545119 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.587946 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:02.672225 51914 executor.go:111] Tasks: 0 done / 88 total; 44 can run W0503 19:25:02.672532 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:02.672586 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:02.672852 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:02.672883 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:02.672975 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:02.673067 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:02.674402 51914 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0503 19:25:02.677288 51914 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0503 19:25:02.677661 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.677834 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.678363 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.679801 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.679901 51914 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0503 19:25:02.680119 51914 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0503 19:25:02.680233 51914 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0503 19:25:02.713356 51914 target.go:112] Terraform output is in /tmp/test1089259790/out --- PASS: TestNvidia (0.20s) === RUN TestMinimalGossip I0503 19:25:02.715521 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:02.715631 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:02.715673 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:02.715706 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:02.715738 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:02.715796 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:02.715819 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:02.715841 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:02.715858 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:02.732753 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.732834 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.841826 51914 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0503 19:25:02.842056 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:02.842410 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:02.843015 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:02.847384 51914 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0503 19:25:02.850843 51914 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0503 19:25:02.851264 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.851464 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.852316 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.852680 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.852754 51914 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0503 19:25:02.852983 51914 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0503 19:25:02.853111 51914 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0503 19:25:02.889958 51914 target.go:112] Terraform output is in /tmp/test4042389431/out --- PASS: TestMinimalGossip (0.18s) === RUN TestMinimalGossipIRSA I0503 19:25:02.892073 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:02.892178 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:02.892223 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:02.892282 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:02.892331 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:02.892399 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:02.892431 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:02.892473 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:02.892497 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:02.909423 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:02.909498 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:03.013153 51914 executor.go:111] Tasks: 0 done / 89 total; 44 can run W0503 19:25:03.014279 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:03.014393 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:03.014825 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:03.017986 51914 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0503 19:25:03.020925 51914 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0503 19:25:03.021272 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:03.021497 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:03.022228 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:03.022798 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:03.022861 51914 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0503 19:25:03.023081 51914 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0503 19:25:03.023206 51914 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0503 19:25:03.057105 51914 target.go:112] Terraform output is in /tmp/test2935947842/out --- PASS: TestMinimalGossipIRSA (0.17s) === RUN TestMinimalGCE I0503 19:25:03.059316 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:03.060388 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0503 19:25:03.943002 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:03.943088 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:03.943175 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0503 19:25:03.943199 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0503 19:25:03.943213 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:03.943238 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:03.943283 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:03.944221 51914 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0503 19:25:03.949679 51914 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0503 19:25:03.952728 51914 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0503 19:25:03.954970 51914 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0503 19:25:03.955350 51914 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0503 19:25:03.975504 51914 target.go:112] Terraform output is in /tmp/test205577732/out --- PASS: TestMinimalGCE (0.92s) === RUN TestMinimalGCEPrivate I0503 19:25:03.978328 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:03.979939 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:04.810923 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:04.811010 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:04.811142 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0503 19:25:04.811172 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0503 19:25:04.811195 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0503 19:25:04.811227 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0503 19:25:04.811289 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:04.812966 51914 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0503 19:25:04.817936 51914 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0503 19:25:04.821035 51914 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0503 19:25:04.822894 51914 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0503 19:25:04.823667 51914 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0503 19:25:04.823762 51914 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0503 19:25:04.851741 51914 target.go:112] Terraform output is in /tmp/test1222840506/out --- PASS: TestMinimalGCEPrivate (0.88s) === RUN TestMinimalGCEInternalLoadBalancer I0503 19:25:04.854736 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:04.856530 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:05.007540 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:05.007611 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:05.007728 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0503 19:25:05.007758 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0503 19:25:05.007779 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0503 19:25:05.007813 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0503 19:25:05.007871 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0503 19:25:05.007912 51914 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0503 19:25:05.009491 51914 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0503 19:25:05.011766 51914 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0503 19:25:05.015022 51914 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0503 19:25:05.015042 51914 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0503 19:25:05.016608 51914 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0503 19:25:05.017369 51914 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0503 19:25:05.017597 51914 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0503 19:25:05.017751 51914 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0503 19:25:05.035369 51914 target.go:112] Terraform output is in /tmp/test2259445930/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.18s) === RUN TestMinimalGCELongClusterName I0503 19:25:05.037263 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:05.038408 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0503 19:25:05.240202 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:05.240275 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:05.240358 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0503 19:25:05.240378 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0503 19:25:05.240393 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0503 19:25:05.240413 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0503 19:25:05.240455 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:05.241356 51914 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0503 19:25:05.243987 51914 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0503 19:25:05.249868 51914 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0503 19:25:05.251357 51914 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0503 19:25:05.251477 51914 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0503 19:25:05.272034 51914 target.go:112] Terraform output is in /tmp/test2081696174/out --- PASS: TestMinimalGCELongClusterName (0.24s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0503 19:25:05.274086 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:05.275227 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0503 19:25:05.479884 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:05.479951 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:05.480049 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0503 19:25:05.480070 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0503 19:25:05.480085 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0503 19:25:05.480112 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0503 19:25:05.480154 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0503 19:25:05.480184 51914 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0503 19:25:05.481148 51914 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0503 19:25:05.483454 51914 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0503 19:25:05.488151 51914 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0503 19:25:05.488194 51914 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0503 19:25:05.491318 51914 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0503 19:25:05.491514 51914 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0503 19:25:05.491676 51914 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0503 19:25:05.491828 51914 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0503 19:25:05.521225 51914 target.go:112] Terraform output is in /tmp/test1083730214/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.25s) === RUN TestHA I0503 19:25:05.524110 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:05.524235 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:05.524283 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:05.524331 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:05.524376 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:05.524462 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:05.524495 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:05.524531 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:05.524557 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:05.558566 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:05.561992 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.562079 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.562132 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.562242 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.602201 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:05.679855 51914 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0503 19:25:05.680263 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:05.680309 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:05.680310 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:05.681346 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:05.681403 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:05.681372 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:05.681481 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:05.682655 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:05.685836 51914 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0503 19:25:05.691088 51914 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0503 19:25:05.691265 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.691396 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.691533 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.691705 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.692278 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.693447 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.693991 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.694183 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:05.694328 51914 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0503 19:25:05.695261 51914 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0503 19:25:05.696112 51914 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0503 19:25:05.741998 51914 target.go:112] Terraform output is in /tmp/test1129702965/out --- PASS: TestHA (0.22s) === RUN TestHighAvailabilityGCE I0503 19:25:05.744644 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:05.745829 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:05.930054 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:05.930168 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:05.930294 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0503 19:25:05.930328 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0503 19:25:05.930352 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0503 19:25:05.930385 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0503 19:25:05.930448 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0503 19:25:05.930532 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0503 19:25:05.930605 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:05.932788 51914 executor.go:111] Tasks: 0 done / 83 total; 48 can run I0503 19:25:05.935114 51914 executor.go:111] Tasks: 48 done / 83 total; 20 can run I0503 19:25:05.939985 51914 executor.go:111] Tasks: 68 done / 83 total; 7 can run I0503 19:25:05.944697 51914 executor.go:111] Tasks: 75 done / 83 total; 5 can run I0503 19:25:05.945551 51914 executor.go:111] Tasks: 80 done / 83 total; 3 can run I0503 19:25:05.945730 51914 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0503 19:25:05.984073 51914 target.go:112] Terraform output is in /tmp/test645492915/out --- PASS: TestHighAvailabilityGCE (0.24s) === RUN TestComplex I0503 19:25:05.988008 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:05.988130 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:05.988182 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:05.988284 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:05.988337 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:05.988379 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:05.988410 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:05.988452 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:05.988481 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:06.009229 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:06.011958 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:06.012030 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.323460 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:07.490954 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0503 19:25:07.491312 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:07.495973 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:07.491351 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:07.496207 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:07.496654 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:07.499911 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:07.503333 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:07.503940 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.504197 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.505154 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.506888 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.507084 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:07.507471 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:07.507621 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0503 19:25:07.560287 51914 target.go:112] Terraform output is in /tmp/test1436892385/out I0503 19:25:07.563578 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:07.563686 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:07.563733 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:07.563776 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:07.563820 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:07.563857 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:07.563884 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:07.563920 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:07.563944 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:07.594675 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:07.598249 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.598328 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.639925 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:07.828899 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0503 19:25:07.829326 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:07.829522 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:07.829545 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:07.829632 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:07.829722 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:07.831342 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:07.838763 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:07.839392 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.839630 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.840571 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.842442 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.842607 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:07.843041 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:07.843237 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0503 19:25:07.906469 51914 target.go:112] Terraform output is in /tmp/test47697499/out --- PASS: TestComplex (1.92s) === RUN TestCompress I0503 19:25:07.909872 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:07.910009 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:07.910060 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:07.910112 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:07.910159 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:07.910247 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:07.910277 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:07.910313 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:07.910338 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:07.935630 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:07.938938 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.939028 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:07.977665 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:08.062661 51914 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0503 19:25:08.062933 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:08.063623 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:08.063674 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:08.063774 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:08.075668 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:08.078315 51914 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0503 19:25:08.081360 51914 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0503 19:25:08.081574 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.081769 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.082934 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.089092 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.089264 51914 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0503 19:25:08.089550 51914 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0503 19:25:08.089688 51914 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0503 19:25:08.130448 51914 target.go:112] Terraform output is in /tmp/test1654354175/out --- PASS: TestCompress (0.22s) === RUN TestExternalPolicies I0503 19:25:08.132515 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:08.132607 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:08.132646 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:08.132677 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:08.132716 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:08.132778 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:08.132796 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:08.132819 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:08.132836 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:08.149515 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:08.151581 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.151643 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.192354 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:08.288145 51914 executor.go:111] Tasks: 0 done / 100 total; 45 can run W0503 19:25:08.288618 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:08.288649 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:08.288843 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:08.288870 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:08.288864 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:08.288976 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:08.292788 51914 executor.go:111] Tasks: 45 done / 100 total; 20 can run I0503 19:25:08.296013 51914 executor.go:111] Tasks: 65 done / 100 total; 30 can run I0503 19:25:08.296522 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.296716 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.297393 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.298213 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.298281 51914 executor.go:111] Tasks: 95 done / 100 total; 3 can run I0503 19:25:08.298534 51914 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0503 19:25:08.298620 51914 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0503 19:25:08.349077 51914 target.go:112] Terraform output is in /tmp/test459437331/out --- PASS: TestExternalPolicies (0.22s) === RUN TestMinimalIPv6 I0503 19:25:08.352433 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:08.352565 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:08.352616 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:08.352669 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:08.352723 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:08.352817 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:08.352849 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:08.352889 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:08.352916 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:08.381381 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:08.384893 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.384982 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.425343 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:08.570504 51914 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0503 19:25:08.570761 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:08.570779 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:08.570778 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:08.570843 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:08.571871 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:08.572663 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:08.575818 51914 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0503 19:25:08.579628 51914 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0503 19:25:08.579983 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.580133 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.580782 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.581573 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.581660 51914 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0503 19:25:08.581975 51914 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0503 19:25:08.582151 51914 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0503 19:25:08.621248 51914 target.go:112] Terraform output is in /tmp/test1906656255/out --- PASS: TestMinimalIPv6 (0.27s) === RUN TestMinimalIPv6Private I0503 19:25:08.624830 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:08.624958 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:08.625011 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:08.625062 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:08.625113 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:08.625202 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:08.625234 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:08.625272 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:08.625301 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:08.649406 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0503 19:25:08.651600 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.651665 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.690921 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:08.845791 51914 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0503 19:25:08.846083 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:08.846104 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:08.846168 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:08.846182 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:08.846267 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:08.846379 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:08.848323 51914 executor.go:111] Tasks: 46 done / 124 total; 23 can run I0503 19:25:08.851706 51914 executor.go:111] Tasks: 69 done / 124 total; 36 can run I0503 19:25:08.851960 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.852105 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.852820 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.853969 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.854054 51914 executor.go:111] Tasks: 105 done / 124 total; 11 can run I0503 19:25:08.854508 51914 executor.go:111] Tasks: 116 done / 124 total; 8 can run I0503 19:25:08.855239 51914 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0503 19:25:08.913516 51914 target.go:112] Terraform output is in /tmp/test337219636/out --- PASS: TestMinimalIPv6Private (0.29s) === RUN TestMinimalIPv6Calico I0503 19:25:08.917576 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:08.917715 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:08.917768 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:08.917864 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:08.917912 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:08.917953 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:08.917980 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:08.918017 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:08.918042 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:08.934576 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:08.937843 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.937958 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:08.980721 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:09.361451 51914 executor.go:111] Tasks: 0 done / 109 total; 47 can run I0503 19:25:09.361960 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:09.361996 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:09.361994 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:09.362097 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:09.362164 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:09.362202 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:09.366729 51914 executor.go:111] Tasks: 47 done / 109 total; 21 can run I0503 19:25:09.369735 51914 executor.go:111] Tasks: 68 done / 109 total; 31 can run I0503 19:25:09.370009 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.370195 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.371190 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.371485 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.371562 51914 executor.go:111] Tasks: 99 done / 109 total; 5 can run I0503 19:25:09.371843 51914 executor.go:111] Tasks: 104 done / 109 total; 5 can run I0503 19:25:09.372091 51914 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0503 19:25:09.407543 51914 target.go:112] Terraform output is in /tmp/test2987382575/out --- PASS: TestMinimalIPv6Calico (0.49s) === RUN TestMinimalIPv6Cilium I0503 19:25:09.410300 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:09.410388 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:09.410427 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:09.410457 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:09.410489 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:09.410546 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:09.410562 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:09.410585 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:09.410602 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:09.427842 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:09.430117 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.430201 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.468674 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:09.604365 51914 executor.go:111] Tasks: 0 done / 107 total; 46 can run W0503 19:25:09.604775 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:09.604845 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:09.604940 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:09.604962 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:09.605042 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:09.605148 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:09.606745 51914 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0503 19:25:09.615112 51914 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0503 19:25:09.615579 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.615839 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.616917 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.617459 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.617543 51914 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0503 19:25:09.617827 51914 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0503 19:25:09.618075 51914 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0503 19:25:09.655295 51914 target.go:112] Terraform output is in /tmp/test172757250/out --- PASS: TestMinimalIPv6Cilium (0.25s) === RUN TestMinimalWarmPool I0503 19:25:09.657629 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:09.657713 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:09.657749 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:09.657777 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:09.657810 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:09.657866 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:09.657882 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:09.657903 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:09.657920 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:09.672509 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:09.674439 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.674495 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.712720 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:09.862483 51914 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0503 19:25:09.864131 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:09.864355 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:09.864383 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:09.864471 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:09.864869 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:09.865917 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:09.867794 51914 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0503 19:25:09.873442 51914 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0503 19:25:09.873857 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.874045 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.874562 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.875563 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.875644 51914 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0503 19:25:09.875813 51914 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0503 19:25:09.875959 51914 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0503 19:25:09.906677 51914 target.go:112] Terraform output is in /tmp/test54883283/out --- PASS: TestMinimalWarmPool (0.25s) === RUN TestMinimalEtcd I0503 19:25:09.910143 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:09.910276 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:09.910325 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:09.910377 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:09.910423 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:09.910509 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:09.910539 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:09.910575 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:09.910599 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:09.936680 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:09.939605 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.939688 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:09.978574 51914 zones.go:93] ListHostedZonesPages { } W0503 19:25:10.038735 51914 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0503 19:25:10.038772 51914 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W0503 19:25:10.039812 51914 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0503 19:25:10.046698 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:10.047146 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:10.047164 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:10.047173 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.047272 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:10.047243 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.047301 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.049619 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:10.053772 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:10.054055 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.054228 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.055003 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.055732 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.055796 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:10.056023 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:10.056148 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:10.087186 51914 target.go:112] Terraform output is in /tmp/test3710588848/out --- PASS: TestMinimalEtcd (0.18s) === RUN TestMinimalGp3 I0503 19:25:10.089320 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:10.089407 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:10.089445 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:10.089477 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:10.089509 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:10.089564 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:10.089581 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:10.089605 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:10.089623 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:10.104110 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:10.105825 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.105879 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.143234 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:10.219960 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:10.220350 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:10.220418 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.220376 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.220451 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:10.220668 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.220657 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:10.222287 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:10.226615 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:10.226895 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.227077 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.228716 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.231127 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.231227 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:10.231506 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:10.231655 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:10.259012 51914 target.go:112] Terraform output is in /tmp/test4073955874/out --- PASS: TestMinimalGp3 (0.17s) === RUN TestMinimalLongClusterName I0503 19:25:10.261043 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:10.261140 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:10.261179 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:10.261213 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:10.261246 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:10.261304 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:10.261321 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:10.261343 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:10.261360 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:10.275251 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:10.276981 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.277038 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0503 19:25:10.314343 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:10.383843 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0503 19:25:10.384371 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.384425 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:10.384573 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.384667 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:10.384695 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.384806 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:10.388952 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:10.391968 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:10.392207 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.392425 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.393060 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.393696 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.393764 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:10.393973 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:10.394088 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:10.426921 51914 target.go:112] Terraform output is in /tmp/test2076648977/out --- PASS: TestMinimalLongClusterName (0.17s) === RUN TestExistingSG I0503 19:25:10.429034 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:10.429129 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:10.429167 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:10.429197 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:10.429229 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:10.429286 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:10.429304 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:10.429330 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:10.429347 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:10.448812 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:10.451186 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.451267 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.451321 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.451374 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.490068 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:10.566288 51914 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0503 19:25:10.566583 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:10.566614 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.566680 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:10.566748 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.566975 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.567071 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.567082 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.567203 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:10.568613 51914 executor.go:111] Tasks: 53 done / 146 total; 26 can run I0503 19:25:10.575172 51914 executor.go:111] Tasks: 79 done / 146 total; 58 can run I0503 19:25:10.575473 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.575602 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.575829 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.576018 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.576403 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.577231 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.578491 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.578740 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.578843 51914 executor.go:111] Tasks: 137 done / 146 total; 5 can run I0503 19:25:10.579202 51914 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0503 19:25:10.579466 51914 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0503 19:25:10.642803 51914 target.go:112] Terraform output is in /tmp/test2121367314/out --- PASS: TestExistingSG (0.22s) === RUN TestBastionAdditionalUserData I0503 19:25:10.645530 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:10.645644 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:10.645694 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:10.645742 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:10.645793 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:10.645861 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:10.645892 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:10.645934 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:10.645959 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:10.665344 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:10.667238 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.667300 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.667337 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.705944 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:10.780100 51914 executor.go:111] Tasks: 0 done / 122 total; 48 can run W0503 19:25:10.780450 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.780497 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:10.780522 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.780618 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:10.780593 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.780600 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.780838 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:10.782214 51914 executor.go:111] Tasks: 48 done / 122 total; 29 can run I0503 19:25:10.785506 51914 executor.go:111] Tasks: 77 done / 122 total; 36 can run I0503 19:25:10.785682 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.785974 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.786125 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.786627 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.786725 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.786959 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.787030 51914 executor.go:111] Tasks: 113 done / 122 total; 5 can run I0503 19:25:10.787346 51914 executor.go:111] Tasks: 118 done / 122 total; 4 can run I0503 19:25:10.787521 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0503 19:25:10.833976 51914 target.go:112] Terraform output is in /tmp/test2822364413/out --- PASS: TestBastionAdditionalUserData (0.19s) === RUN TestPrivateWeave I0503 19:25:10.836306 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:10.836397 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:10.836434 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:10.836464 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:10.836497 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:10.836556 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:10.836572 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:10.836594 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:10.836610 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:10.852975 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:10.854830 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.854886 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.854919 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.898542 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:10.980806 51914 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0503 19:25:10.981380 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:10.981399 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.981420 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:10.981479 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:10.981443 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:10.981522 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:10.981575 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:10.984234 51914 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0503 19:25:10.989969 51914 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0503 19:25:10.990154 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.990386 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.990487 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.990592 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.990954 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.991685 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:10.991762 51914 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0503 19:25:10.992048 51914 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0503 19:25:10.992212 51914 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0503 19:25:11.037354 51914 target.go:112] Terraform output is in /tmp/test1471407392/out --- PASS: TestPrivateWeave (0.20s) === RUN TestPrivateFlannel I0503 19:25:11.039635 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:11.039722 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:11.039761 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:11.039826 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:11.039858 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:11.039885 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:11.039903 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:11.039928 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:11.039945 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:11.056766 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:11.058706 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.058766 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.058800 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.099438 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:11.247735 51914 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0503 19:25:11.247950 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:11.247955 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:11.248382 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:11.248533 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:11.248546 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:11.248565 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:11.248651 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:11.249988 51914 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0503 19:25:11.255424 51914 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0503 19:25:11.255649 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.255858 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.255984 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.256052 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.256436 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.257443 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.257508 51914 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0503 19:25:11.257794 51914 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0503 19:25:11.257975 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:11.303624 51914 target.go:112] Terraform output is in /tmp/test1804699308/out --- PASS: TestPrivateFlannel (0.27s) === RUN TestPrivateCalico I0503 19:25:11.307134 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:11.307252 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:11.307301 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:11.307394 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:11.307440 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:11.307478 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:11.307505 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:11.307543 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:11.307568 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:11.330271 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:11.332346 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.332409 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.332445 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.370534 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:11.693119 51914 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0503 19:25:11.693454 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:11.693631 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:11.693651 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:11.693732 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:11.693798 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:11.693805 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:11.695943 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:11.698497 51914 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0503 19:25:11.701492 51914 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0503 19:25:11.701773 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.701888 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.702021 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.702112 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.702730 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.703261 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.703327 51914 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0503 19:25:11.703603 51914 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0503 19:25:11.703765 51914 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0503 19:25:11.749136 51914 target.go:112] Terraform output is in /tmp/test1326581033/out --- PASS: TestPrivateCalico (0.44s) === RUN TestPrivateCilium I0503 19:25:11.752159 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:11.752266 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:11.752307 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:11.752341 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:11.752375 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:11.752438 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:11.752456 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:11.752480 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:11.752497 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:11.769567 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:11.771618 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.771678 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.771712 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.810200 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:11.905508 51914 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0503 19:25:11.905896 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:11.905880 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:11.905922 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:11.906023 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:11.907479 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:11.907725 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:11.907898 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:11.910905 51914 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0503 19:25:11.914299 51914 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0503 19:25:11.914707 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.914919 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.915137 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.915239 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.915732 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.917656 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:11.917762 51914 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0503 19:25:11.918060 51914 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0503 19:25:11.918240 51914 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0503 19:25:11.967984 51914 target.go:112] Terraform output is in /tmp/test2323144564/out --- PASS: TestPrivateCilium (0.22s) === RUN TestPrivateCilium2 I0503 19:25:11.971314 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:11.971436 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:11.971487 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:11.971579 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:11.971625 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:11.971666 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:11.971692 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:11.971727 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:11.971752 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:11.999138 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:12.002309 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.002394 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.002451 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.041741 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:12.526407 51914 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0503 19:25:12.526868 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:12.526987 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:12.527074 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:12.527099 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:12.527193 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:12.527194 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:12.527250 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:12.530205 51914 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0503 19:25:12.536230 51914 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0503 19:25:12.536453 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.536588 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.536665 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.536808 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.537372 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.538412 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.538494 51914 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0503 19:25:12.538826 51914 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0503 19:25:12.539041 51914 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0503 19:25:12.589339 51914 target.go:112] Terraform output is in /tmp/test3570893029/out --- PASS: TestPrivateCilium2 (0.62s) === RUN TestPrivateCiliumAdvanced I0503 19:25:12.594582 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:12.594700 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:12.594754 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:12.594805 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:12.594856 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:12.594943 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:12.594972 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:12.595008 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:12.595037 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:12.620830 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:12.623106 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.623170 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.623204 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.663114 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:12.808265 51914 executor.go:111] Tasks: 0 done / 126 total; 53 can run W0503 19:25:12.808703 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:12.808835 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:12.808885 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:12.810311 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:12.811398 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:12.811527 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:12.811635 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:12.815283 51914 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0503 19:25:12.819074 51914 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0503 19:25:12.819485 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.819733 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.819982 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.820103 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.820804 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.821097 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.821175 51914 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0503 19:25:12.821438 51914 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0503 19:25:12.821616 51914 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0503 19:25:12.881927 51914 target.go:112] Terraform output is in /tmp/test1964450939/out --- PASS: TestPrivateCiliumAdvanced (0.29s) === RUN TestPrivateCanal I0503 19:25:12.886044 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:12.886183 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:12.886239 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:12.886344 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:12.886397 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:12.886439 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:12.886470 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:12.886507 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:12.886538 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:12.910839 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:12.914499 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.914633 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.914698 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:12.954032 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:13.293848 51914 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0503 19:25:13.294662 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:13.295433 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:13.295459 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:13.295563 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:13.297036 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.297294 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.297461 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.300121 51914 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0503 19:25:13.303576 51914 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0503 19:25:13.303820 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.303957 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.304157 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.304331 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.304932 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.305377 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:13.305451 51914 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0503 19:25:13.305729 51914 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0503 19:25:13.305898 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:13.351004 51914 target.go:112] Terraform output is in /tmp/test2743303261/out --- PASS: TestPrivateCanal (0.47s) === RUN TestPrivateKopeio I0503 19:25:13.353894 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:13.353987 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:13.354024 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:13.354055 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:13.354087 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:13.354144 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:13.354160 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:13.354184 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:13.354201 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:13.372515 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:13.374463 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.374522 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.374557 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.413336 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:13.491263 51914 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0503 19:25:13.491552 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:13.491586 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:13.491681 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:13.491706 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.491842 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:13.491906 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.492025 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.495845 51914 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0503 19:25:13.498881 51914 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0503 19:25:13.499064 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.499390 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.499557 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.499686 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.500422 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.501017 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.501082 51914 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0503 19:25:13.501364 51914 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0503 19:25:13.501527 51914 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0503 19:25:13.564764 51914 target.go:112] Terraform output is in /tmp/test2476899119/out --- PASS: TestPrivateKopeio (0.21s) === RUN TestUnmanaged I0503 19:25:13.567019 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:13.567106 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:13.567143 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:13.567176 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:13.567209 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:13.567268 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:13.567285 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:13.567308 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:13.567325 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:13.584801 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:13.586748 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.586805 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.586842 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.625981 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:13.715367 51914 executor.go:111] Tasks: 0 done / 108 total; 45 can run W0503 19:25:13.715966 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.715965 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:13.716082 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.716176 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.716412 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:13.716436 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:13.716518 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:13.718244 51914 executor.go:111] Tasks: 45 done / 108 total; 25 can run I0503 19:25:13.722610 51914 executor.go:111] Tasks: 70 done / 108 total; 31 can run I0503 19:25:13.722927 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.723029 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.723165 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.723404 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.725006 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.726951 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.727045 51914 executor.go:111] Tasks: 101 done / 108 total; 4 can run I0503 19:25:13.727351 51914 executor.go:111] Tasks: 105 done / 108 total; 3 can run I0503 19:25:13.727575 51914 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0503 19:25:13.772406 51914 target.go:112] Terraform output is in /tmp/test1756363142/out --- PASS: TestUnmanaged (0.21s) === RUN TestPrivateSharedSubnet I0503 19:25:13.774741 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:13.774829 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:13.774867 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:13.774898 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:13.774930 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:13.774987 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:13.775004 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:13.775027 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:13.775043 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:13.791760 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:13.793686 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.793742 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.793774 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.834955 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:13.926928 51914 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0503 19:25:13.927150 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:13.927174 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:13.927258 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:13.927263 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.927344 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:13.927556 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:13.927643 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:13.929452 51914 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0503 19:25:13.929711 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:13.933855 51914 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0503 19:25:13.934023 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.934232 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.934320 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.934359 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.936003 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.937737 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:13.937847 51914 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0503 19:25:13.938101 51914 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0503 19:25:13.938245 51914 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0503 19:25:13.976854 51914 target.go:112] Terraform output is in /tmp/test4171904822/out --- PASS: TestPrivateSharedSubnet (0.21s) === RUN TestPrivateSharedIP I0503 19:25:13.980096 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:13.980218 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:13.980266 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:13.980313 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:13.980359 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:13.980442 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:13.980473 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:13.980508 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:13.980534 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:14.005604 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:14.008380 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.008458 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.008517 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.046623 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:14.157371 51914 executor.go:111] Tasks: 0 done / 116 total; 45 can run W0503 19:25:14.157621 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.157867 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:14.157903 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:14.158089 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.158154 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:14.158186 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:14.158292 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:14.159581 51914 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0503 19:25:14.159792 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:14.162591 51914 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0503 19:25:14.162714 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.162929 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.164059 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.164236 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.164315 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.166633 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:14.166724 51914 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0503 19:25:14.166999 51914 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0503 19:25:14.167212 51914 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0503 19:25:14.219123 51914 target.go:112] Terraform output is in /tmp/test932901741/out --- PASS: TestPrivateSharedIP (0.24s) === RUN TestPrivateDns1 I0503 19:25:14.221316 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:14.221408 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:14.221447 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:14.221479 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:14.221513 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:14.221573 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:14.221589 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:14.221612 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:14.221631 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:14.240451 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.240510 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.240546 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.343278 51914 executor.go:111] Tasks: 0 done / 120 total; 46 can run W0503 19:25:14.343641 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.345271 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:14.345760 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:14.345904 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.348410 51914 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0503 19:25:14.348736 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0503 19:25:14.348765 51914 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0503 19:25:14.348820 51914 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0503 19:25:14.348830 51914 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0503 19:25:14.348870 51914 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0503 19:25:14.349259 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.349355 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.352030 51914 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0503 19:25:14.352103 51914 api.go:53] PageSize not implemented W0503 19:25:14.352127 51914 api.go:53] PageSize not implemented I0503 19:25:14.354781 51914 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0503 19:25:14.354994 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.355197 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.356047 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.356962 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.357072 51914 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0503 19:25:14.357292 51914 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0503 19:25:14.357418 51914 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0503 19:25:14.427660 51914 target.go:112] Terraform output is in /tmp/test840672752/out --- PASS: TestPrivateDns1 (0.21s) === RUN TestPrivateDns2 I0503 19:25:14.429918 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:14.430009 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:14.430046 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:14.430077 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:14.430109 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:14.430167 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:14.430183 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:14.430208 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:14.430224 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:14.451558 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.451637 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.451688 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.554910 51914 executor.go:111] Tasks: 0 done / 116 total; 44 can run W0503 19:25:14.555349 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:14.555382 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:14.557639 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.558176 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:14.561705 51914 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0503 19:25:14.561950 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:14.562098 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0503 19:25:14.562136 51914 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0503 19:25:14.562215 51914 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0503 19:25:14.562227 51914 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0503 19:25:14.562284 51914 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0503 19:25:14.562802 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.562952 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.565809 51914 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0503 19:25:14.565942 51914 api.go:53] PageSize not implemented W0503 19:25:14.566008 51914 api.go:53] PageSize not implemented I0503 19:25:14.571246 51914 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0503 19:25:14.571436 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.571660 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.572987 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.573836 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.573946 51914 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0503 19:25:14.574264 51914 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0503 19:25:14.574462 51914 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0503 19:25:14.634246 51914 target.go:112] Terraform output is in /tmp/test473389351/out --- PASS: TestPrivateDns2 (0.21s) === RUN TestDiscoveryFeatureGate I0503 19:25:14.637796 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:14.637923 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:14.637978 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:14.638077 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:14.638131 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:14.638176 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:14.638207 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:14.638247 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:14.638275 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:14.664984 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:14.668248 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.668338 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.709907 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:14.862199 51914 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0503 19:25:14.862428 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:14.862620 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.862609 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:14.862670 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:14.862652 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:14.862710 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:14.870530 51914 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0503 19:25:14.875829 51914 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0503 19:25:14.876054 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.876389 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.877193 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.879155 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.879276 51914 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0503 19:25:14.879818 51914 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0503 19:25:14.880081 51914 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0503 19:25:14.939856 51914 target.go:112] Terraform output is in /tmp/test4198080951/out --- PASS: TestDiscoveryFeatureGate (0.31s) === RUN TestVFSServiceAccountIssuerDiscovery I0503 19:25:14.943269 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:14.943373 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:14.943415 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:14.943453 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:14.943490 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:14.943557 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:14.943578 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:14.943605 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:14.943625 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:14.962950 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:14.964865 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:14.964927 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.004101 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:15.091321 51914 executor.go:111] Tasks: 0 done / 90 total; 45 can run W0503 19:25:15.092596 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:15.093127 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:15.093158 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:15.093273 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:15.093938 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:15.095266 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:15.096965 51914 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0503 19:25:15.099841 51914 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0503 19:25:15.099975 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.100221 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.100670 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.101586 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.101651 51914 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0503 19:25:15.101826 51914 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0503 19:25:15.101932 51914 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0503 19:25:15.134940 51914 target.go:112] Terraform output is in /tmp/test2406930189/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.19s) === RUN TestAWSLBController I0503 19:25:15.137095 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:15.137184 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:15.137221 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:15.137253 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:15.137285 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:15.137348 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:15.137365 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:15.137387 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:15.137404 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:15.152434 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:15.154256 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.154307 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.192540 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:15.620344 51914 executor.go:111] Tasks: 0 done / 96 total; 49 can run W0503 19:25:15.620898 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:15.621429 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:15.621471 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:15.621494 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:15.621526 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:15.621672 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:15.625238 51914 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0503 19:25:15.633050 51914 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0503 19:25:15.633645 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.634597 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.636881 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.639574 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.639700 51914 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0503 19:25:15.640013 51914 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0503 19:25:15.640221 51914 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0503 19:25:15.683896 51914 target.go:112] Terraform output is in /tmp/test1806544728/out --- PASS: TestAWSLBController (0.55s) === RUN TestManyAddons I0503 19:25:15.688852 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:15.688985 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:15.689035 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:15.689112 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:15.689149 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:15.689176 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:15.689200 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:15.689224 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:15.689249 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:15.708307 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:15.710449 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.710514 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:15.752428 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:16.386673 51914 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0503 19:25:16.387025 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:16.387457 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:16.389541 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:16.390076 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:16.391757 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:16.391911 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:16.394594 51914 executor.go:111] Tasks: 50 done / 95 total; 19 can run I0503 19:25:16.399690 51914 executor.go:111] Tasks: 69 done / 95 total; 22 can run I0503 19:25:16.400055 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.400284 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.400943 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.404228 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.404399 51914 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0503 19:25:16.405148 51914 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0503 19:25:16.405971 51914 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0503 19:25:16.448443 51914 target.go:112] Terraform output is in /tmp/test576537477/out --- PASS: TestManyAddons (0.76s) === RUN TestManyAddonsCCMIRSA I0503 19:25:16.453509 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:16.453604 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:16.453644 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:16.453706 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:16.453742 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:16.453772 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:16.453789 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:16.453813 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:16.453831 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:16.482692 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:16.486618 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.486733 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:16.525618 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:17.246206 51914 executor.go:111] Tasks: 0 done / 111 total; 59 can run I0503 19:25:17.248936 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:17.249925 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:17.249988 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:17.250230 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:17.251080 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:17.251218 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:17.252164 51914 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0503 19:25:17.257632 51914 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0503 19:25:17.257855 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.257986 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.258620 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.259346 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.259412 51914 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0503 19:25:17.259598 51914 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0503 19:25:17.259718 51914 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0503 19:25:17.298985 51914 target.go:112] Terraform output is in /tmp/test1640955575/out --- PASS: TestManyAddonsCCMIRSA (0.85s) === RUN TestManyAddonsCCMIRSA23 I0503 19:25:17.304236 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:17.304339 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:17.304385 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:17.304463 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:17.304501 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:17.304532 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:17.304548 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:17.304577 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:17.304601 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:17.320713 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0503 19:25:17.322956 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.323021 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:17.362593 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:18.074373 51914 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0503 19:25:18.075131 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:18.075231 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:18.075231 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:18.075271 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:18.075371 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:18.075424 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:18.078336 51914 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0503 19:25:18.084892 51914 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0503 19:25:18.085363 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.085538 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.086106 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.088144 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.088276 51914 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0503 19:25:18.088556 51914 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0503 19:25:18.088743 51914 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0503 19:25:18.138327 51914 target.go:112] Terraform output is in /tmp/test1765413170/out --- PASS: TestManyAddonsCCMIRSA23 (0.84s) === RUN TestManyAddonsCCMIRSA24 I0503 19:25:18.143498 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:18.143589 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:18.143629 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:18.143693 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:18.143726 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:18.143753 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:18.143770 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:18.143791 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:18.143808 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:18.160035 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:18.162151 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.162207 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.201956 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:18.908852 51914 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0503 19:25:18.909446 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:18.909510 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:18.909772 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:18.909798 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:18.909896 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:18.910940 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:18.917052 51914 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0503 19:25:18.920074 51914 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0503 19:25:18.920454 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.920662 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.921356 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.923036 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:18.923163 51914 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0503 19:25:18.923429 51914 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0503 19:25:18.923574 51914 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0503 19:25:18.976453 51914 target.go:112] Terraform output is in /tmp/test2794264934/out --- PASS: TestManyAddonsCCMIRSA24 (0.84s) === RUN TestManyAddonsCCMIRSA25 I0503 19:25:18.988615 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:18.988765 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:18.988824 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:18.989321 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:18.989376 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:18.989418 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:18.989449 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:18.989488 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:18.989516 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:19.017882 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:19.021609 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.021700 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.064315 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:19.795516 51914 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0503 19:25:19.799012 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:19.799051 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:19.799358 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:19.800104 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:19.800131 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:19.800236 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:19.802063 51914 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0503 19:25:19.808287 51914 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0503 19:25:19.808572 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.808776 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.809731 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.810294 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.810364 51914 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0503 19:25:19.810634 51914 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0503 19:25:19.810773 51914 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0503 19:25:19.859236 51914 target.go:112] Terraform output is in /tmp/test2706271474/out --- PASS: TestManyAddonsCCMIRSA25 (0.88s) === RUN TestManyAddonsCCMIRSA26 I0503 19:25:19.866839 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:19.866980 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:19.867032 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:19.867127 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:19.867172 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:19.867210 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:19.867236 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:19.867272 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:19.867297 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:19.893726 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:19.897118 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.897204 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:19.941331 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:20.599757 51914 executor.go:111] Tasks: 0 done / 111 total; 59 can run W0503 19:25:20.600840 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:20.601315 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:20.601364 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:20.601458 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:20.602481 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:20.602843 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:20.606653 51914 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0503 19:25:20.609708 51914 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0503 19:25:20.609928 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.610087 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.610768 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.613270 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.614239 51914 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0503 19:25:20.614712 51914 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0503 19:25:20.615395 51914 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0503 19:25:20.655893 51914 target.go:112] Terraform output is in /tmp/test2533178235/out --- PASS: TestManyAddonsCCMIRSA26 (0.80s) === RUN TestCCM I0503 19:25:20.663828 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:20.663925 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:20.663966 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:20.664032 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:20.664063 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:20.664091 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:20.664109 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:20.664131 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:20.664149 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:20.685509 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:20.688625 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.688713 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:20.727614 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:21.251504 51914 executor.go:111] Tasks: 0 done / 96 total; 51 can run W0503 19:25:21.251829 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:21.253358 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.253953 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:21.254023 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:21.254047 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:21.254119 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:21.258416 51914 executor.go:111] Tasks: 51 done / 96 total; 19 can run I0503 19:25:21.262934 51914 executor.go:111] Tasks: 70 done / 96 total; 22 can run I0503 19:25:21.263300 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.263467 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.265244 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.266804 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.267307 51914 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0503 19:25:21.267566 51914 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0503 19:25:21.267746 51914 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0503 19:25:21.302965 51914 target.go:112] Terraform output is in /tmp/test2952700517/out --- PASS: TestCCM (0.64s) === RUN TestExternalDNS I0503 19:25:21.308721 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:21.308868 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:21.308933 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:21.308980 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:21.309024 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:21.309118 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:21.309146 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:21.309180 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:21.309204 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:21.335125 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:21.336873 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.336934 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.375801 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:21.475516 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:21.475831 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:21.475971 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.476057 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:21.476072 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:21.476140 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:21.477935 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.481310 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:21.484426 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:21.484745 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.484892 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.485571 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.486090 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.486154 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:21.486358 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:21.486481 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:21.517462 51914 target.go:112] Terraform output is in /tmp/test3631675508/out --- PASS: TestExternalDNS (0.21s) === RUN TestExternalDNSIRSA I0503 19:25:21.519626 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:21.519715 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:21.519754 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:21.519786 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:21.519818 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:21.519875 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:21.519893 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:21.519929 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:21.519956 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:21.536312 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:21.538092 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.538148 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.577237 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:21.660364 51914 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0503 19:25:21.660673 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:21.660739 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.660893 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:21.660918 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:21.660978 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:21.661004 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.665727 51914 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0503 19:25:21.669013 51914 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0503 19:25:21.669202 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.669314 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.669861 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.671599 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.671696 51914 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0503 19:25:21.671922 51914 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0503 19:25:21.672046 51914 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0503 19:25:21.711812 51914 target.go:112] Terraform output is in /tmp/test141371374/out --- PASS: TestExternalDNSIRSA (0.19s) === RUN TestKarpenter I0503 19:25:21.714263 51914 featureflag.go:160] FeatureFlag "Karpenter"=true I0503 19:25:21.714292 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:21.714375 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:21.714413 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:21.714476 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:21.714509 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:21.714534 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:21.714551 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:21.714575 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:21.714593 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:21.735132 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:21.737332 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.737388 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.737422 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.777510 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:21.899518 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.899639 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.899733 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.972466 51914 executor.go:111] Tasks: 0 done / 108 total; 53 can run W0503 19:25:21.972721 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:21.972990 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:21.973082 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:21.973258 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:21.973295 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:21.975022 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:21.978038 51914 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0503 19:25:21.981112 51914 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0503 19:25:21.981266 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.981427 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.981570 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.981771 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.981963 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.982370 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.982454 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.983019 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:21.983095 51914 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0503 19:25:21.983354 51914 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0503 19:25:21.983486 51914 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0503 19:25:22.024874 51914 target.go:112] Terraform output is in /tmp/test3038673081/out I0503 19:25:22.028139 51914 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.31s) === RUN TestSharedSubnet I0503 19:25:22.028234 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.028323 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.028362 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.028392 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.028426 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.028484 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.028501 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.028523 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.028540 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.042799 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.044507 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.044564 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.084432 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:22.154820 51914 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0503 19:25:22.155290 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:22.155314 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:22.155398 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:22.155475 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:22.155503 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.155633 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:22.156836 51914 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0503 19:25:22.157021 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:22.159699 51914 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0503 19:25:22.159877 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.160058 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.160729 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.161242 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.161309 51914 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0503 19:25:22.161505 51914 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0503 19:25:22.161636 51914 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0503 19:25:22.190210 51914 target.go:112] Terraform output is in /tmp/test2007328924/out --- PASS: TestSharedSubnet (0.16s) === RUN TestSharedVPC I0503 19:25:22.192305 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.192395 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.192434 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.192466 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.192499 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.192558 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.192577 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.192600 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.192617 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.206828 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.208558 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.208615 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.246941 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:22.364529 51914 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0503 19:25:22.364848 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:22.364882 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:22.364997 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:22.365020 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.365055 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:22.365361 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.367554 51914 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0503 19:25:22.367985 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:22.371786 51914 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0503 19:25:22.372078 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.372212 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.373233 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.373481 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.373548 51914 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0503 19:25:22.373801 51914 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0503 19:25:22.373940 51914 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0503 19:25:22.406605 51914 target.go:112] Terraform output is in /tmp/test376304217/out --- PASS: TestSharedVPC (0.22s) === RUN TestExistingIAM I0503 19:25:22.408612 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.408702 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.408740 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.408775 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.408807 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.408867 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.408886 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.408911 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.408945 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.428351 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.430402 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.430461 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.430498 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.430531 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.469278 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:22.543550 51914 executor.go:111] Tasks: 0 done / 101 total; 51 can run W0503 19:25:22.543894 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:22.543934 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.543979 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:22.543995 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:22.544054 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:22.544219 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:22.544232 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.544425 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:22.545757 51914 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0503 19:25:22.551706 51914 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0503 19:25:22.551991 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.552063 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.552156 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.552270 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.552656 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.553331 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.553408 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.553652 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:22.553726 51914 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0503 19:25:22.553978 51914 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0503 19:25:22.554138 51914 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0503 19:25:22.598351 51914 target.go:112] Terraform output is in /tmp/test371602415/out --- PASS: TestExistingIAM (0.19s) === RUN TestPhaseNetwork I0503 19:25:22.600900 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.601001 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.601042 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.601075 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.601108 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.601136 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.601155 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.601178 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.601195 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.625566 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.627604 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.627671 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.627707 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.668878 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:22.742168 51914 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0503 19:25:22.742424 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:22.742505 51914 executor.go:111] Tasks: 46 done / 119 total; 28 can run I0503 19:25:22.742798 51914 executor.go:111] Tasks: 74 done / 119 total; 36 can run I0503 19:25:22.743029 51914 executor.go:111] Tasks: 110 done / 119 total; 5 can run I0503 19:25:22.743148 51914 executor.go:111] Tasks: 115 done / 119 total; 4 can run I0503 19:25:22.743241 51914 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0503 19:25:22.748296 51914 target.go:112] Terraform output is in /tmp/test124549355/out --- PASS: TestPhaseNetwork (0.15s) === RUN TestExternalLoadBalancer I0503 19:25:22.748662 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.748746 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.748779 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.748809 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.748841 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.748897 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.748931 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.748958 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.748976 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.766470 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.770105 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.770235 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.810943 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:22.881569 51914 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0503 19:25:22.882126 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:22.882184 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:22.882201 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:22.882225 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:22.882209 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:22.882313 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:22.883700 51914 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0503 19:25:22.889452 51914 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0503 19:25:22.889843 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.889920 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.890598 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.891111 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.891178 51914 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0503 19:25:22.891399 51914 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0503 19:25:22.891521 51914 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0503 19:25:22.927161 51914 target.go:112] Terraform output is in /tmp/test35721801/out --- PASS: TestExternalLoadBalancer (0.18s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0503 19:25:22.931846 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:22.931951 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:22.931990 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:22.932026 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:22.932059 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:22.932120 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:22.932141 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:22.932164 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:22.932182 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:22.951499 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:22.953479 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953536 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953571 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953604 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953637 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953665 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.953695 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:22.992828 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.068759 51914 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0503 19:25:23.068989 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.069115 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:23.069188 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:23.069346 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.069432 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:23.069494 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.069455 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.069541 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:23.070891 51914 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0503 19:25:23.077239 51914 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0503 19:25:23.077446 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.077546 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.077668 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.077820 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.078647 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.079213 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.079287 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.079348 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.079419 51914 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0503 19:25:23.079703 51914 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0503 19:25:23.079862 51914 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0503 19:25:23.124144 51914 target.go:112] Terraform output is in /tmp/test657022476/out --- PASS: TestMixedInstancesASG (0.20s) === RUN TestMixedInstancesSpotASG I0503 19:25:23.128367 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:23.128467 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:23.128504 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:23.128569 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:23.128601 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:23.128627 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:23.128643 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:23.128667 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:23.128684 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:23.148335 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:23.150342 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150399 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150434 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150466 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150494 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150520 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.150546 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.189272 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.264739 51914 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0503 19:25:23.265025 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:23.265128 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:23.265142 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.265195 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:23.265400 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.265229 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:23.265438 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.265534 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:23.266875 51914 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0503 19:25:23.273073 51914 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0503 19:25:23.273436 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.273602 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.273790 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.273979 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.274072 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.274954 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.275025 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.275219 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:23.275286 51914 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0503 19:25:23.275602 51914 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0503 19:25:23.275765 51914 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0503 19:25:23.321117 51914 target.go:112] Terraform output is in /tmp/test3340813930/out --- PASS: TestMixedInstancesSpotASG (0.20s) === RUN TestAdditionalObjects I0503 19:25:23.323899 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:23.323991 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:23.324029 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:23.324060 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:23.324092 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:23.324148 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:23.324165 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:23.324187 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:23.324204 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:23.340333 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0503 19:25:23.342221 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.342295 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.382079 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.485972 51914 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0503 19:25:23.486445 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.486445 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:23.486753 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:23.486795 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.486782 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.486870 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:23.488458 51914 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0503 19:25:23.491941 51914 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0503 19:25:23.492271 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.492354 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.493386 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.493796 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0503 19:25:23.493866 51914 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0503 19:25:23.494089 51914 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0503 19:25:23.494205 51914 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0503 19:25:23.525695 51914 target.go:112] Terraform output is in /tmp/test1548869198/out --- PASS: TestAdditionalObjects (0.20s) === RUN TestContainerd I0503 19:25:23.527855 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:23.527945 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:23.527982 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:23.528017 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:23.528049 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:23.528107 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:23.528125 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:23.528147 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:23.528163 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:23.542505 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:23.544211 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.544268 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.583010 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.653775 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0503 19:25:23.654075 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.654443 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:23.654467 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.654530 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:23.654540 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.654601 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:23.658755 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:23.662224 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:23.662596 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.662786 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.663566 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.664119 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.664181 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:23.664336 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:23.664461 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:23.695370 51914 target.go:112] Terraform output is in /tmp/test3259598979/out --- PASS: TestContainerd (0.17s) === RUN TestContainerdCustom I0503 19:25:23.697492 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:23.697584 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:23.697622 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:23.697653 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:23.697685 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:23.697747 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:23.697766 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:23.697790 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:23.697808 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:23.713428 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:23.715291 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.715351 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.754637 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.824538 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0503 19:25:23.825011 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.825098 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:23.825166 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.825121 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.825201 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:23.825263 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:23.826712 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:23.829816 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:23.830159 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.830326 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.831018 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.831580 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.831644 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:23.831824 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:23.831943 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:23.861892 51914 target.go:112] Terraform output is in /tmp/test1461833683/out --- PASS: TestContainerdCustom (0.17s) === RUN TestDockerCustom I0503 19:25:23.863949 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:23.864372 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:23.864410 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:23.864441 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:23.864472 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:23.864530 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:23.864547 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:23.864571 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:23.864588 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:23.879913 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:23.881834 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.881890 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.920478 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:23.989993 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:23.990235 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0503 19:25:23.990524 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.990636 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0503 19:25:23.990687 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:23.990660 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:23.990743 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0503 19:25:23.993351 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:23.997747 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:23.998094 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.998266 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.999110 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.999576 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:23.999643 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:23.999862 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:23.999981 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:24.030475 51914 target.go:112] Terraform output is in /tmp/test4288439435/out --- PASS: TestDockerCustom (0.17s) === RUN TestAPIServerNodes I0503 19:25:24.032525 51914 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0503 19:25:24.032544 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:24.032627 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:24.032665 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:24.032699 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:24.032732 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:24.032794 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:24.032811 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:24.032834 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:24.032850 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:24.048542 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0503 19:25:24.050404 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:24.050460 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:24.050495 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.168715 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:25.279220 51914 executor.go:111] Tasks: 0 done / 100 total; 47 can run W0503 19:25:25.279557 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:25.279548 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:25.279703 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:25.279732 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:25.279814 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:25.279824 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:25.279831 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:25.281484 51914 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0503 19:25:25.284452 51914 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0503 19:25:25.284586 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.284735 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.284847 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.285246 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.285416 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.287111 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.287198 51914 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0503 19:25:25.287417 51914 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0503 19:25:25.287600 51914 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0503 19:25:25.323129 51914 target.go:112] Terraform output is in /tmp/test2485694397/out I0503 19:25:25.325430 51914 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.29s) === RUN TestNTHQueueProcessor I0503 19:25:25.325522 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:25.325618 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:25.325656 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:25.325688 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:25.325720 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:25.325777 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:25.325794 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:25.325818 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:25.325835 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:25.342569 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0503 19:25:25.344290 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:25.344350 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.749109 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:26.832549 51914 executor.go:111] Tasks: 0 done / 101 total; 45 can run W0503 19:25:26.832746 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:26.832832 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:26.832849 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:26.832907 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:26.833205 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:26.833284 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:26.834984 51914 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0503 19:25:26.840421 51914 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0503 19:25:26.841035 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.841250 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.842648 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.843622 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.843705 51914 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0503 19:25:26.843947 51914 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0503 19:25:26.844120 51914 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0503 19:25:26.872204 51914 target.go:112] Terraform output is in /tmp/test2733738383/out --- PASS: TestNTHQueueProcessor (1.55s) === RUN TestCustomIRSA I0503 19:25:26.874707 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:26.874788 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:26.874819 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:26.874848 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:26.874879 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:26.874934 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:26.874951 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:26.874974 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:26.874991 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:26.891226 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0503 19:25:26.893111 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.893163 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:26.934659 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:27.458318 51914 executor.go:111] Tasks: 0 done / 100 total; 50 can run W0503 19:25:27.458695 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:27.458853 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:27.459935 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:27.459972 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:27.460073 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:27.460757 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:27.463681 51914 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0503 19:25:27.466766 51914 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0503 19:25:27.466978 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.467145 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.467770 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.468332 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.468394 51914 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0503 19:25:27.468598 51914 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0503 19:25:27.468701 51914 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0503 19:25:27.512098 51914 target.go:112] Terraform output is in /tmp/test789784216/out --- PASS: TestCustomIRSA (0.64s) === RUN TestClusterNameDigit I0503 19:25:27.519229 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.519372 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:27.519424 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:27.519474 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:27.519524 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:27.519614 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:27.519646 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:27.519683 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:27.519709 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:27.544400 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0503 19:25:27.546273 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.546331 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.587643 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:27.674357 51914 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0503 19:25:27.674666 51914 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0503 19:25:27.674701 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:27.674789 51914 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0503 19:25:27.674932 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target W0503 19:25:27.675018 51914 warmpool.go:124] ASG warm pool is not supported by the terraform target I0503 19:25:27.675097 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:27.676850 51914 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0503 19:25:27.682893 51914 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0503 19:25:27.683811 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.684032 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.685010 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.686346 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.686459 51914 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0503 19:25:27.686749 51914 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0503 19:25:27.686905 51914 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0503 19:25:27.741520 51914 target.go:112] Terraform output is in /tmp/test2925724463/out --- PASS: TestClusterNameDigit (0.23s) === RUN TestLifecycleMinimalAWS I0503 19:25:27.749315 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.749436 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:27.749480 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:27.749513 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:27.749546 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:27.749575 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:27.749593 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:27.749615 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:27.749632 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:27.749676 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0503 19:25:27.768219 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:27.769951 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.770016 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0503 19:25:27.811855 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:27.893930 51914 iamrole.go:89] ListRoles: { } I0503 19:25:27.896040 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:27.896417 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:27.896490 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0503 19:25:27.896519 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.896606 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0503 19:25:27.896781 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.896895 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:27.896936 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.896958 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0503 19:25:27.897059 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0503 19:25:27.897101 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.897128 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.897337 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:27.897419 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:27.897580 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.897780 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:27.897844 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:27.897881 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:27.897934 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:27.897972 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:27.898115 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:27.898163 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.898328 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:27.898453 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:27.898552 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:27.898670 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.898745 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0503 19:25:27.898805 51914 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0503 19:25:27.898770 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0503 19:25:27.898849 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0503 19:25:27.898983 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0503 19:25:27.898993 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:27.899079 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:27.899111 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:27.899137 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.899183 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:27.899208 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0503 19:25:27.899264 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:27.899344 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:27.899338 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:27.899332 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0503 19:25:27.899480 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:27.899574 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:27.899610 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0503 19:25:27.899726 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:27.899777 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:27.899805 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:27.899827 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:27.899954 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:27.899946 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0503 19:25:27.899991 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0503 19:25:27.900064 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:27.900097 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:27.900126 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:27.900949 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0503 19:25:27.902925 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:27.903046 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:27.903123 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.903265 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.903416 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.903495 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.903661 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.903929 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:27.904066 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:27.904114 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:27.904206 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:27.904304 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0503 19:25:27.904441 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:27.904473 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.904541 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:27.904588 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.904729 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0503 19:25:27.904858 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.904982 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I0503 19:25:27.905109 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:27.905233 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0503 19:25:27.905347 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.905475 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.905642 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.905770 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.905897 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0503 19:25:27.907596 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:27.908086 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:27.908330 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:27.908408 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:27.908466 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:27.908561 51914 dns.go:238] Pre-creating DNS records I0503 19:25:27.908610 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:27.913931 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:27.916706 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:27.916792 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0503 19:25:27.954718 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:28.061507 51914 iamrole.go:89] ListRoles: { } I0503 19:25:28.062972 51914 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0503 19:25:28.063273 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:28.063293 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.063465 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.063628 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:28.063668 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:28.063723 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:28.063773 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.063956 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0503 19:25:28.064076 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:28.064135 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:28.065234 51914 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0503 19:25:28.065441 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.065550 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0503 19:25:28.065539 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:28.065606 51914 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0503 19:25:28.065620 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:28.065683 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.065770 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:28.065794 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.065870 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0503 19:25:28.065970 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:28.066001 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0503 19:25:28.066077 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:28.069981 51914 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0503 19:25:28.070164 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:28.070278 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:28.070343 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:28.070437 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:28.070570 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:28.070778 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:28.070956 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:28.071889 51914 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0503 19:25:28.072111 51914 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0503 19:25:28.072276 51914 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0503 19:25:28.073817 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:28.073857 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:28.073877 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:28.073895 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:28.073912 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:28.073934 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:28.073952 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:28.073971 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:28.073989 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:28.074008 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:28.074029 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:28.074052 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:28.074074 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:28.074092 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:28.074110 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:28.074129 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:28.074148 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:28.074167 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:28.074188 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:28.074208 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:28.074226 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:28.074244 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:28.074262 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:28.074282 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:28.074302 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:28.074321 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:28.075053 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:28.075070 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:28.075098 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075129 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0503 19:25:28.075162 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075197 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075227 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075261 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075289 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075311 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0503 19:25:28.075339 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075371 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0503 19:25:28.075396 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0503 19:25:28.075421 51914 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0503 19:25:28.075441 51914 api.go:53] PageSize not implemented I0503 19:25:28.075454 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:28.075473 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:28.075484 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:28.075508 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:28.075519 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:28.075552 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:28.075571 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:28.075613 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:28.075639 51914 iamrole.go:89] ListRoles: { } I0503 19:25:28.075679 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:28.075706 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:28.075725 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.minimal.example.com ok autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0503 19:25:28.076102 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0503 19:25:28.076158 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok keypair:key-1 ok I0503 19:25:28.076141 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0503 19:25:28.076197 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-config:lt-2 ok I0503 19:25:28.076224 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0503 19:25:28.076238 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } volume:vol-2 ok I0503 19:25:28.076270 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0503 19:25:28.076318 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0503 19:25:28.076126 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0503 19:25:28.076292 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0503 19:25:28.076389 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:28.076405 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:28.076430 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:28.076458 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0503 19:25:28.076575 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:28.076584 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:28.076606 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:28.076640 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:28.076690 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:28.076704 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0503 19:25:28.076757 51914 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0503 19:25:28.076768 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0503 19:25:28.076790 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0503 19:25:28.076811 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0503 19:25:28.076836 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0503 19:25:28.076843 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0503 19:25:28.076860 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0503 19:25:28.076899 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:28.076944 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:28.076983 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:28.077165 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.33s) === RUN TestLifecycleMinimalOpenstack I0503 19:25:28.077328 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0503 19:25:28.086881 51914 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:28.247702 51914 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0503 19:25:28.247746 51914 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0503 19:25:28.257468 51914 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0503 19:25:28.264603 51914 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0503 19:25:28.310353 51914 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0503 19:25:28.312630 51914 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0503 19:25:28.314864 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:28.314877 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:28.314866 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0503 19:25:28.317999 51914 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:28.469345 51914 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0503 19:25:28.469397 51914 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0503 19:25:28.485267 51914 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0503 19:25:28.487896 51914 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0503 19:25:28.495876 51914 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0503 19:25:28.497140 51914 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0503 19:25:28.499141 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:28.499327 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:28.499376 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0503 19:25:28.501209 51914 executor.go:111] Tasks: 96 done / 96 total; 0 can run ServerGroup:3caded49-bdc9-4da8-8adb-ce34fbd05ae7 ok ServerGroup:9d7550f2-3ff1-4663-ac38-14ba74397b86 ok Network:a8d7f24e-2c87-4d61-a3d4-226fe9b0b7c8 ok Subnet:0bf61fb0-812f-4303-bfa3-0a1742366f81 ok Instance:107a4779-71b2-45fc-9623-7698f73af4f1 ok SecurityGroup:42d44a65-55dc-4faa-a263-4996d26b63b1 ok Port:fc0b32c0-e300-4f91-a8b7-c51124ffacab ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:9350b619-1b25-49c8-b1aa-d0340126f7b9 ok Port:ca768776-1a42-46a4-921c-5156654c3077 ok SecurityGroup:86d0c113-ac91-4abb-a2fa-9042dac67895 ok Instance:657bb0d4-9134-4244-8c0f-c05cce5611c9 ok Instance:47ebb0a8-6c97-4988-a8b2-726f14763850 ok SecurityGroup:c31ea9ed-9aea-4e6d-a962-567946a50e5a ok --- PASS: TestLifecycleMinimalOpenstack (1.53s) === RUN TestLifecycleMinimalGCE I0503 19:25:29.603524 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0503 19:25:29.605280 51914 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0503 19:25:29.793279 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:29.793471 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:29.793556 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0503 19:25:29.793578 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0503 19:25:29.793592 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:29.793617 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:29.793672 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:29.794637 51914 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0503 19:25:29.796773 51914 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0503 19:25:29.796819 51914 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0503 19:25:29.800074 51914 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0503 19:25:29.801590 51914 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0503 19:25:29.801729 51914 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0503 19:25:29.801794 51914 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0503 19:25:29.969755 51914 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0503 19:25:29.969941 51914 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0503 19:25:29.970084 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0503 19:25:29.970147 51914 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0503 19:25:29.970196 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:29.970269 51914 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0503 19:25:29.970405 51914 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0503 19:25:29.971478 51914 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0503 19:25:29.979687 51914 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0503 19:25:29.982794 51914 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0503 19:25:29.984923 51914 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0503 19:25:29.985029 51914 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0503 19:25:29.985998 51914 gce.go:98] Scanning zones: [us-test1-a] I0503 19:25:29.986237 51914 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0503 19:25:29.986253 51914 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" Disk:d1-etcd-main-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1683141929 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1683141929 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.38s) === RUN TestLifecycleFloatingIPOpenstack I0503 19:25:29.987118 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0503 19:25:29.993585 51914 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:30.144896 51914 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0503 19:25:30.144935 51914 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0503 19:25:30.156526 51914 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0503 19:25:30.159832 51914 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0503 19:25:30.221263 51914 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0503 19:25:30.223239 51914 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0503 19:25:30.224773 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:30.225113 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0503 19:25:30.229280 51914 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0503 19:25:30.372568 51914 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0503 19:25:30.372609 51914 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0503 19:25:30.387580 51914 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0503 19:25:30.390346 51914 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0503 19:25:30.398223 51914 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0503 19:25:30.399447 51914 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0503 19:25:30.401061 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:30.402041 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0503 19:25:30.402277 51914 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0503 19:25:30.402783 51914 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:c541f10b-13ec-46ab-b87d-a067508ff220 ok Port:e9594710-594e-4ea2-97df-0098c522f379 ok Router:2030927c-5c96-4121-ae1d-231e898b9f71 ok Instance:1fd8e2e8-bf77-4b83-9a52-ebaacce9980d ok Instance:50f66794-e770-4a70-9f60-2fe19dd6fb8c ok Port:5bb86bcb-7f5b-45c6-917e-bf77840afb08 ok Port:a4136d7a-e193-4e29-becf-ec8954af67e9 ok SecurityGroup:78247fbb-0218-43ee-987a-f0bf22d442e6 ok Instance:a8ab6f86-ced1-4827-a4e2-f74cbaa970e7 ok Subnet:c541f10b-13ec-46ab-b87d-a067508ff220 ok ServerGroup:0dae9554-7b86-4a41-aa52-71f1866d9293 ok Port:c4d6771a-6c05-432d-9606-a3b89106529a ok Network:6e9cbbf9-d40a-402c-aca6-7a2d722268a1 ok ServerGroup:4c3612e6-4248-4418-a3e2-9cb7bf02420c ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:7829f2cf-a660-411b-9de3-352c79356e94 ok SecurityGroup:18b84d79-fad3-4b8d-b05b-be2026c367aa ok --- PASS: TestLifecycleFloatingIPOpenstack (1.53s) === RUN TestLifecyclePrivateCalico I0503 19:25:31.513559 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:31.513682 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:31.513732 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:31.513778 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:31.513825 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:31.513861 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:31.513899 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:31.513939 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:31.513965 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:31.514032 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0503 19:25:31.535877 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:31.537936 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.538001 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.538038 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.579713 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:31.927144 51914 iamrole.go:89] ListRoles: { } I0503 19:25:31.928460 51914 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0503 19:25:31.928619 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0503 19:25:31.930142 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:31.930234 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0503 19:25:31.930324 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" }] } I0503 19:25:31.930279 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:31.930419 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.930695 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.930868 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:31.930946 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.931089 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:31.931150 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.931344 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:31.931333 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0503 19:25:31.931395 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:31.931406 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0503 19:25:31.931719 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:31.932543 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:31.932585 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.932936 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0503 19:25:31.933021 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" }] } I0503 19:25:31.932970 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:31.933083 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:31.933184 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.933278 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:31.933358 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:31.934019 51914 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0503 19:25:31.934141 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0503 19:25:31.934136 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.934217 51914 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0503 19:25:31.934206 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0503 19:25:31.934246 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0503 19:25:31.934255 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.934322 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0503 19:25:31.934299 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:31.934355 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:31.934357 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0503 19:25:31.934385 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.934447 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:31.934426 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0503 19:25:31.934480 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0503 19:25:31.934510 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0503 19:25:31.934605 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:31.934667 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:31.934695 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:31.934721 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] } I0503 19:25:31.934679 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0503 19:25:31.934769 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.934937 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0503 19:25:31.935055 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:31.935094 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:31.935125 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0503 19:25:31.935249 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:31.935318 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.935369 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:31.935426 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0503 19:25:31.935499 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:31.935548 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:31.935482 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0503 19:25:31.935607 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0503 19:25:31.935672 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0503 19:25:31.935737 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:31.935769 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:31.935793 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:31.935867 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:31.935905 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:31.935934 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:31.936022 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:31.935761 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0503 19:25:31.936078 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:31.936132 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:31.936174 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:31.936208 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:31.936259 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:31.936299 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:31.936421 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0503 19:25:31.936445 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:31.936609 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:31.936691 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:31.936777 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:31.936863 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0503 19:25:31.938594 51914 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0503 19:25:31.938789 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:31.938889 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0503 19:25:31.938980 51914 api.go:53] PageSize not implemented W0503 19:25:31.939010 51914 api.go:53] PageSize not implemented I0503 19:25:31.939045 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:31.939147 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0503 19:25:31.939213 51914 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.939251 51914 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0503 19:25:31.939182 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.939317 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0503 19:25:31.939346 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0503 19:25:31.939347 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0503 19:25:31.939386 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0503 19:25:31.939396 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0503 19:25:31.939465 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0503 19:25:31.939424 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0503 19:25:31.939520 51914 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0503 19:25:31.939576 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0503 19:25:31.939606 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0503 19:25:31.939562 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0503 19:25:31.939647 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0503 19:25:31.939768 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.939895 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.940016 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:31.940159 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.940262 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.940336 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0503 19:25:31.940449 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.940586 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.940718 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:31.940849 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:31.940902 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0503 19:25:31.941029 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" }] }] } I0503 19:25:31.941154 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0503 19:25:31.941270 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.941415 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0503 19:25:31.941541 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.941687 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.941768 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.941883 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0503 19:25:31.941973 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:31.942051 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:31.942118 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:31.942189 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I0503 19:25:31.942316 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.942451 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.942561 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.942636 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.942953 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0503 19:25:31.943036 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:31.943122 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.943189 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:31.943215 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:31.943236 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:31.943304 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.943379 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.943454 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.943538 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0503 19:25:31.943668 51914 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0503 19:25:31.943736 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:31.943832 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:31.943920 51914 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:31.943852 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:31.944025 51914 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0503 19:25:31.944089 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:31.944123 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:31.944160 51914 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0503 19:25:31.944231 51914 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0503 19:25:31.944289 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:31.944382 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0503 19:25:31.944424 51914 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0503 19:25:31.944452 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:31.944476 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:31.944509 51914 dns.go:238] Pre-creating DNS records I0503 19:25:31.944525 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:31.948824 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0503 19:25:31.953350 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.953417 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.953456 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:31.991965 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.310389 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.311692 51914 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0503 19:25:32.311953 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.312193 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.312211 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:32.312281 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.312404 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.312560 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:32.312580 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:32.312613 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.312639 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.312732 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.313792 51914 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0503 19:25:32.313972 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0503 19:25:32.314069 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0503 19:25:32.313891 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0503 19:25:32.314173 51914 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0503 19:25:32.314151 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0503 19:25:32.314311 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.314369 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:32.314420 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.314505 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0503 19:25:32.314635 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:32.314681 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.314810 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:32.314846 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0503 19:25:32.314977 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.315024 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0503 19:25:32.315154 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:32.315207 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.315294 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:32.315320 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.315398 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.315462 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.315516 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.317542 51914 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0503 19:25:32.317904 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0503 19:25:32.318028 51914 api.go:53] PageSize not implemented I0503 19:25:32.318038 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:32.318098 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0503 19:25:32.318068 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.318164 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0503 19:25:32.318224 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.318284 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0503 19:25:32.318276 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0503 19:25:32.318354 51914 api.go:53] PageSize not implemented I0503 19:25:32.318388 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.318519 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0503 19:25:32.318448 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.318566 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0503 19:25:32.318683 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.318953 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.319779 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.319857 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.319958 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.320094 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.320673 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.320814 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.321821 51914 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0503 19:25:32.321932 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0503 19:25:32.322006 51914 api.go:53] PageSize not implemented I0503 19:25:32.322034 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:32.322013 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.322040 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.322126 51914 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0503 19:25:32.322155 51914 api.go:53] PageSize not implemented I0503 19:25:32.322192 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.322374 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0503 19:25:32.322457 51914 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0503 19:25:32.322584 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.322658 51914 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0503 19:25:32.324755 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:32.324792 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:32.324823 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:32.324857 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:32.324891 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:32.324928 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0503 19:25:32.324964 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:32.324998 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:32.325032 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:32.325065 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:32.325101 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:32.325138 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:32.325177 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:32.325219 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0503 19:25:32.325263 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:32.325298 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:32.325332 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:32.325365 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:32.325404 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:32.325438 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:32.325475 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:32.325517 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:32.325555 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:32.325603 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:32.325642 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:32.325678 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.325728 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:32.325764 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:32.325810 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:32.325831 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:32.325849 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:32.325870 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:32.325890 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:32.325910 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:32.326861 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:32.326878 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:32.326904 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.326946 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0503 19:25:32.326992 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327028 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327058 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327088 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327113 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327141 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0503 19:25:32.327178 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327211 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:32.327228 51914 natgateway.go:117] DescribeNatGateways: { } I0503 19:25:32.327250 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0503 19:25:32.327276 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0503 19:25:32.327308 51914 api.go:53] PageSize not implemented I0503 19:25:32.327319 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0503 19:25:32.327351 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:32.327366 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:32.327378 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:32.327392 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:32.327403 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:32.327434 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.327452 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.327480 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:32.327508 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.327538 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:32.327569 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:32.327593 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0503 19:25:32.327615 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:32.327635 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0503 19:25:32.328021 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0503 19:25:32.328061 51914 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } I0503 19:25:32.328104 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } nat-gateway:nat-1 ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok iam-instance-profile:masters.privatecalico.example.com ok I0503 19:25:32.328130 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok autoscaling-group:nodes.privatecalico.example.com ok I0503 19:25:32.328021 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.328220 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0503 19:25:32.328139 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:bastion.privatecalico.example.com ok load-balancer:api-privatecalico-example-0uch4k ok I0503 19:25:32.328143 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0503 19:25:32.328288 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0503 19:25:32.328422 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.328444 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:32.328366 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0503 19:25:32.328491 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0503 19:25:32.328499 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } keypair:key-1 ok I0503 19:25:32.328584 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0503 19:25:32.328603 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:32.328664 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0503 19:25:32.328802 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.328816 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0503 19:25:32.328841 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0503 19:25:32.328890 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.328937 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0503 19:25:32.328991 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.328999 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:32.329022 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-2 ok I0503 19:25:32.329120 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:32.329185 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:32.329262 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0503 19:25:32.329270 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I0503 19:25:32.329295 51914 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com subnet:subnet-2 ok I0503 19:25:32.329304 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0503 19:25:32.329311 51914 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0503 19:25:32.329324 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0503 19:25:32.329381 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0503 19:25:32.329416 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0503 19:25:32.329349 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0503 19:25:32.329424 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } subnet:subnet-1 ok I0503 19:25:32.329446 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0503 19:25:32.329449 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.329473 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0503 19:25:32.329499 51914 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0503 19:25:32.329506 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0503 19:25:32.329525 51914 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0503 19:25:32.329482 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 } ] } W0503 19:25:32.329573 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.329582 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0503 19:25:32.329600 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:32.329627 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0503 19:25:32.329681 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.329689 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0503 19:25:32.329739 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:32.329776 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0503 19:25:32.329823 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:32.329870 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:32.330052 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.82s) === RUN TestLifecyclePrivateKopeio I0503 19:25:32.330202 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:32.330292 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:32.330319 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:32.330342 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:32.330380 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:32.330413 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:32.330430 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:32.330453 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:32.330468 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:32.330504 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0503 19:25:32.353231 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:32.355173 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.355234 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.355272 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.394675 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.478238 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.479596 51914 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0503 19:25:32.480110 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.480178 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.480220 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:32.480274 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0503 19:25:32.480284 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.480369 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0503 19:25:32.480506 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0503 19:25:32.480560 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0503 19:25:32.480476 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.480629 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0503 19:25:32.480697 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0503 19:25:32.480700 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.480867 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:32.480973 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:32.481008 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.481114 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0503 19:25:32.481140 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.481305 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.481428 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:32.481473 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:32.481566 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:32.481631 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:32.481652 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:32.481667 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:32.481690 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.481743 51914 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0503 19:25:32.481811 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0503 19:25:32.481876 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0503 19:25:32.481932 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.481991 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0503 19:25:32.482063 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0503 19:25:32.482105 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0503 19:25:32.482031 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0503 19:25:32.482135 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0503 19:25:32.482119 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0503 19:25:32.482297 51914 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0503 19:25:32.482219 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.482337 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:32.482428 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:32.482431 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0503 19:25:32.482409 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0503 19:25:32.482525 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0503 19:25:32.482593 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0503 19:25:32.482721 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:32.482778 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:32.482806 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:32.482830 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.482883 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0503 19:25:32.482951 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0503 19:25:32.482953 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.483075 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0503 19:25:32.483090 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.483145 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0503 19:25:32.483209 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:32.483220 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0503 19:25:32.483263 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:32.483291 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.483312 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0503 19:25:32.483387 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0503 19:25:32.483202 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0503 19:25:32.483409 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.483476 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:32.483521 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0503 19:25:32.483542 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.483663 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:32.483688 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.483790 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.483838 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.483884 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.483934 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:32.483989 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:32.484046 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:32.484009 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0503 19:25:32.484074 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:32.484104 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0503 19:25:32.484130 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0503 19:25:32.484150 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:32.484176 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:32.484199 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:32.484226 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.484251 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0503 19:25:32.484274 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:32.484300 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:32.484327 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:32.484354 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:32.484388 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:32.484412 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.484438 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:32.484458 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:32.484482 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0503 19:25:32.484523 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:32.484547 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] } I0503 19:25:32.484593 51914 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0503 19:25:32.484788 51914 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0503 19:25:32.484904 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } W0503 19:25:32.485015 51914 api.go:53] PageSize not implemented I0503 19:25:32.484990 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0503 19:25:32.485097 51914 api.go:53] PageSize not implemented I0503 19:25:32.485077 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0503 19:25:32.485166 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0503 19:25:32.485153 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.485244 51914 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0503 19:25:32.485383 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0503 19:25:32.485377 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:32.485426 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0503 19:25:32.485477 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:32.485516 51914 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0503 19:25:32.485547 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:32.485484 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0503 19:25:32.485578 51914 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0503 19:25:32.485630 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:32.485646 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.485669 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0503 19:25:32.485727 51914 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0503 19:25:32.485796 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.485758 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0503 19:25:32.485826 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0503 19:25:32.485840 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.485876 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0503 19:25:32.485964 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.486042 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.486134 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0503 19:25:32.486202 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0503 19:25:32.486285 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.486370 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.486461 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.486535 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.486633 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.486717 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0503 19:25:32.486747 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0503 19:25:32.486772 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.486849 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:32.486871 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:32.486900 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0503 19:25:32.486960 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.487063 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0503 19:25:32.487156 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.487256 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.487354 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:32.487394 51914 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0503 19:25:32.487427 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.487457 51914 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0503 19:25:32.487490 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0503 19:25:32.487580 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0503 19:25:32.487605 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.487697 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:32.487775 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.487871 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:32.487912 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0503 19:25:32.487980 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0503 19:25:32.488049 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.488140 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.488234 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:32.488274 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0503 19:25:32.488367 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.488436 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.488501 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.488567 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0503 19:25:32.488602 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0503 19:25:32.488630 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0503 19:25:32.488663 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0503 19:25:32.488822 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.488943 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.489036 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.489428 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0503 19:25:32.489467 51914 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0503 19:25:32.490669 51914 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0503 19:25:32.490752 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.490764 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.490837 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0503 19:25:32.490911 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.490964 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0503 19:25:32.491010 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I0503 19:25:32.491069 51914 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0503 19:25:32.491190 51914 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0503 19:25:32.491229 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.491260 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.491308 51914 dns.go:238] Pre-creating DNS records I0503 19:25:32.491332 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.499683 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:32.502969 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.503050 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.503114 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.545285 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.622714 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.624292 51914 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0503 19:25:32.624556 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.624767 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.624850 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.624993 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.625090 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:32.625119 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:32.625164 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.625310 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:32.625424 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.625731 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.633012 51914 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0503 19:25:32.633148 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0503 19:25:32.633164 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.633297 51914 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0503 19:25:32.633320 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0503 19:25:32.633282 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.633439 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0503 19:25:32.633596 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.633642 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0503 19:25:32.633787 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:32.633833 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] }] } I0503 19:25:32.633940 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0503 19:25:32.633974 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.634045 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0503 19:25:32.634146 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:32.634187 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.634277 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.634367 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:32.634403 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.634499 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.634600 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0503 19:25:32.634696 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.634736 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0503 19:25:32.634832 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:32.634862 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.634940 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.635013 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:32.635054 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:32.636021 51914 executor.go:111] Tasks: 78 done / 126 total; 39 can run W0503 19:25:32.636095 51914 api.go:53] PageSize not implemented I0503 19:25:32.636116 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.636144 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.636186 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.636228 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0503 19:25:32.636250 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0503 19:25:32.636311 51914 api.go:53] PageSize not implemented I0503 19:25:32.636308 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.636330 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.636397 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0503 19:25:32.636432 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0503 19:25:32.636449 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.636547 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.636674 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.636770 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0503 19:25:32.636976 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.637080 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.637198 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.637393 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.637477 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0503 19:25:32.637588 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.637710 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:32.639305 51914 executor.go:111] Tasks: 117 done / 126 total; 6 can run W0503 19:25:32.639382 51914 api.go:53] PageSize not implemented I0503 19:25:32.639373 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.639401 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.639459 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.639519 51914 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0503 19:25:32.639487 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0503 19:25:32.639539 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0503 19:25:32.639580 51914 api.go:53] PageSize not implemented I0503 19:25:32.639591 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.639695 51914 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0503 19:25:32.639794 51914 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0503 19:25:32.642077 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:32.642115 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:32.642137 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:32.642157 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:32.642176 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:32.642198 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:32.642217 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:32.642236 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:32.642270 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:32.642293 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:32.642314 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:32.642337 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:32.642359 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0503 19:25:32.642385 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:32.642405 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:32.642424 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:32.642445 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:32.642463 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:32.642484 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0503 19:25:32.642506 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:32.642528 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:32.642549 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:32.642569 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:32.642591 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:32.642611 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.642633 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:32.642652 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:32.642673 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0503 19:25:32.642694 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0503 19:25:32.642716 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:32.642736 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:32.642755 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:32.642776 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:32.642797 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:32.642818 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:32.643781 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:32.643799 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:32.643823 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.643860 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0503 19:25:32.643906 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.643941 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.643970 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.644002 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.644026 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.644054 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0503 19:25:32.644093 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.644131 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:32.644149 51914 natgateway.go:117] DescribeNatGateways: { } I0503 19:25:32.644172 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0503 19:25:32.644196 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0503 19:25:32.644224 51914 api.go:53] PageSize not implemented I0503 19:25:32.644234 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0503 19:25:32.644264 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:32.644279 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:32.644290 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:32.644304 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:32.644316 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:32.644348 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.644367 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.644397 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:32.644421 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.644449 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:32.644478 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:32.644506 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1","rtb-3"] } I0503 19:25:32.644530 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0503 19:25:32.644547 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } autoscaling-group:nodes.privatekopeio.example.com ok I0503 19:25:32.644916 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0503 19:25:32.644963 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0503 19:25:32.644970 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:32.645007 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } autoscaling-group:bastion.privatekopeio.example.com ok I0503 19:25:32.644990 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } iam-instance-profile:nodes.privatekopeio.example.com ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0503 19:25:32.645036 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0503 19:25:32.645083 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0503 19:25:32.645109 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0503 19:25:32.645021 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } I0503 19:25:32.645110 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0503 19:25:32.645058 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.645175 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok W0503 19:25:32.645161 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:bastions.privatekopeio.example.com ok I0503 19:25:32.645201 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0503 19:25:32.645227 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0503 19:25:32.645268 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:32.645312 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0503 19:25:32.645344 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0503 19:25:32.645378 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:32.645411 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0503 19:25:32.645497 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.645506 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0503 19:25:32.645524 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:32.645551 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0503 19:25:32.645611 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.645620 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:32.645684 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0503 19:25:32.645698 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0503 19:25:32.645718 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0503 19:25:32.645752 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0503 19:25:32.645782 51914 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0503 19:25:32.645780 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.645798 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0503 19:25:32.645833 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } I0503 19:25:32.645811 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } iam-role:masters.privatekopeio.example.com ok W0503 19:25:32.645859 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.645867 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:32.645878 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0503 19:25:32.645886 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:32.645897 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0503 19:25:32.645913 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.645926 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0503 19:25:32.645945 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0503 19:25:32.645978 51914 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0503 19:25:32.645939 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0503 19:25:32.645993 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:32.645987 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0503 19:25:32.646001 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0503 19:25:32.646005 51914 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0503 19:25:32.646021 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0503 19:25:32.646074 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0503 19:25:32.646104 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0503 19:25:32.646128 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:32.646173 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:32.646215 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:32.646378 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.32s) === RUN TestLifecycleIPv6 I0503 19:25:32.646497 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:32.646587 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:32.646612 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:32.646635 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:32.646665 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:32.646688 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:32.646704 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:32.646724 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:32.646739 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:32.646774 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0503 19:25:32.661933 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:32.663940 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.663997 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.701760 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.845694 51914 iamrole.go:89] ListRoles: { } I0503 19:25:32.846921 51914 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0503 19:25:32.847214 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.847358 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:32.847395 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.847471 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.847490 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0503 19:25:32.847618 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0503 19:25:32.847671 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.847769 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0503 19:25:32.847794 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:32.847828 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0503 19:25:32.847853 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:32.847888 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:32.847925 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:32.847968 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.848008 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.848064 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:32.848206 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.848419 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.848563 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:32.848659 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:32.848692 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.848797 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:32.848864 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:32.849355 51914 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0503 19:25:32.849447 51914 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0503 19:25:32.849499 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0503 19:25:32.849461 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0503 19:25:32.849549 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0503 19:25:32.849637 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0503 19:25:32.849713 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0503 19:25:32.849602 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0503 19:25:32.849783 51914 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0503 19:25:32.849866 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:32.849957 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:32.849994 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:32.850020 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.850129 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0503 19:25:32.850206 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:32.850241 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0503 19:25:32.850372 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:32.850447 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:32.850473 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:32.850344 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0503 19:25:32.850495 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.850568 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:32.850646 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0503 19:25:32.850651 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.850805 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:32.850858 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.850964 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:32.851038 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:32.850918 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0503 19:25:32.851142 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:32.851187 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0503 19:25:32.851304 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0503 19:25:32.851329 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:32.851387 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:32.851415 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:32.851438 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0503 19:25:32.851492 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:32.851518 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0503 19:25:32.852356 51914 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0503 19:25:32.852462 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.852542 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.852606 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.852673 51914 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I0503 19:25:32.852763 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0503 19:25:32.852879 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.853015 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0503 19:25:32.853076 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.853160 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:32.853274 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0503 19:25:32.853346 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.853481 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.853518 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:32.853599 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0503 19:25:32.853685 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0503 19:25:32.853779 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.853820 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.853857 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:32.853904 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:32.853934 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.854008 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0503 19:25:32.854068 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.854154 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:32.854229 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.854307 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.854393 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:32.854424 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.854487 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.854557 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:32.854616 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:32.854658 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0503 19:25:32.854758 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.854860 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0503 19:25:32.854956 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.855048 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.855134 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.855218 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.855280 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.856787 51914 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0503 19:25:32.856841 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.856932 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.856944 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:32.857012 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:32.857016 51914 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.857083 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:32.857123 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0503 19:25:32.857108 51914 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0503 19:25:32.857137 51914 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0503 19:25:32.857188 51914 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0503 19:25:32.857228 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:32.857250 51914 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0503 19:25:32.857266 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:32.857303 51914 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0503 19:25:32.857378 51914 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0503 19:25:32.857438 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:32.857482 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.857556 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0503 19:25:32.857583 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.857646 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.857716 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:32.857777 51914 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0503 19:25:32.857809 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:32.857827 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.857858 51914 dns.go:238] Pre-creating DNS records I0503 19:25:32.857871 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:32.862396 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:32.866147 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.866222 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:32.905612 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.045400 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.046575 51914 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0503 19:25:33.046765 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.047044 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.047172 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.047248 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.047332 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.047461 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:33.047492 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:33.047548 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.047581 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.047712 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.049013 51914 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0503 19:25:33.049134 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.049249 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.049338 51914 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0503 19:25:33.049327 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.049406 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0503 19:25:33.049393 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0503 19:25:33.049480 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0503 19:25:33.049464 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0503 19:25:33.049614 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.049657 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0503 19:25:33.049786 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:33.049841 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0503 19:25:33.049980 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.050102 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.050178 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.052641 51914 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0503 19:25:33.052751 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.052852 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.052934 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.053054 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.053114 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.053161 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:33.053184 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0503 19:25:33.053206 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0503 19:25:33.053240 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.053615 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.053890 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.054023 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.054128 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.054248 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.056334 51914 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0503 19:25:33.056414 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:33.056464 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:33.056475 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0503 19:25:33.056434 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.056499 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0503 19:25:33.056519 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0503 19:25:33.056525 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:33.056547 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0503 19:25:33.056569 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0503 19:25:33.056588 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.057330 51914 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0503 19:25:33.057439 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:33.057547 51914 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0503 19:25:33.057507 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.057576 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:33.057603 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0503 19:25:33.057638 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:33.057737 51914 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0503 19:25:33.057766 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:33.057789 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0503 19:25:33.057898 51914 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0503 19:25:33.061292 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:33.061356 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:33.061378 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:33.061397 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:33.061418 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:33.061438 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0503 19:25:33.061459 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:33.061477 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:33.061500 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:33.061531 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:33.061566 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:33.061597 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:33.061636 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:33.061673 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:33.061701 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:33.061730 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:33.061756 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:33.061786 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:33.061815 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:33.061846 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:33.061875 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:33.061904 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:33.061935 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:33.061969 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:33.061995 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:33.062021 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:33.062046 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:33.062076 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:33.062109 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:33.062140 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:33.063455 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:33.063477 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:33.063516 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063565 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0503 19:25:33.063620 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063671 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063714 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063757 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063793 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063836 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0503 19:25:33.063881 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.063921 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:33.063947 51914 natgateway.go:117] DescribeNatGateways: { } I0503 19:25:33.063986 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0503 19:25:33.064019 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0503 19:25:33.064069 51914 api.go:53] PageSize not implemented I0503 19:25:33.064089 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:33.064114 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:33.064130 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0503 19:25:33.064169 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:33.064190 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:33.064208 51914 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0503 19:25:33.064297 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.064326 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.064369 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:33.064403 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.064441 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:33.064483 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:33.064519 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0503 19:25:33.064551 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:33.064583 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0503 19:25:33.064953 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0503 19:25:33.064949 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0503 19:25:33.065000 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0503 19:25:33.065020 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.065045 51914 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0503 19:25:33.065116 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0503 19:25:33.065149 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok iam-instance-profile:nodes.minimal-ipv6.example.com ok keypair:key-1 ok I0503 19:25:33.065107 51914 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0503 19:25:33.065071 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/kops-controller.internal.minimal-ipv6.example.com. ok I0503 19:25:33.065084 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0503 19:25:33.065443 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:33.065465 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:33.065510 51914 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0503 19:25:33.065577 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:33.065642 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.065686 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:33.065707 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:33.065733 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0503 19:25:33.065820 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-2 ok I0503 19:25:33.065881 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:33.065950 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:33.065990 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:33.066055 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0503 19:25:33.066155 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0503 19:25:33.066269 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0503 19:25:33.066322 51914 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0503 19:25:33.066338 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0503 19:25:33.066274 51914 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0503 19:25:33.066372 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0503 19:25:33.066414 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:33.066450 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0503 19:25:33.066494 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0503 19:25:33.066503 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0503 19:25:33.066522 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0503 19:25:33.066573 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0503 19:25:33.066625 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:33.066678 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:33.066862 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.42s) === RUN TestLifecycleSharedVPC I0503 19:25:33.067004 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:33.067095 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:33.067121 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:33.067145 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:33.067172 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:33.067200 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:33.067218 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:33.067240 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:33.067255 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:33.067288 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0503 19:25:33.082660 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:33.084443 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.084500 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.122726 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.231506 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.233036 51914 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0503 19:25:33.233409 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:33.233506 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:33.233534 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:33.239713 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0503 19:25:33.239870 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0503 19:25:33.239906 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.240073 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0503 19:25:33.240227 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" }] } I0503 19:25:33.240431 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.241180 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:33.241321 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:33.241374 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] }] } I0503 19:25:33.241731 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.241782 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.241852 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:33.242015 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:33.242060 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0503 19:25:33.242526 51914 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0503 19:25:33.242687 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0503 19:25:33.242645 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0503 19:25:33.242709 51914 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0503 19:25:33.242729 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0503 19:25:33.242869 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0503 19:25:33.242923 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:33.242963 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0503 19:25:33.243024 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:33.243078 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.243116 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:33.243065 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0503 19:25:33.243215 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.243381 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0503 19:25:33.243375 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:33.243512 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:33.243580 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.243670 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:33.243769 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:33.243783 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0503 19:25:33.243832 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.243889 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.243961 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:33.244011 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I0503 19:25:33.244591 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0503 19:25:33.245801 51914 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0503 19:25:33.245883 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.246021 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0503 19:25:33.246088 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.246523 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.246797 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.246928 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:33.246957 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:33.247060 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.247208 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.247272 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0503 19:25:33.247378 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.247476 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0503 19:25:33.247561 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.247629 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.247689 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.247814 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0503 19:25:33.247855 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0503 19:25:33.247972 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.248055 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.248128 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.248216 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0503 19:25:33.248288 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.248363 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0503 19:25:33.248450 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.248513 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.249903 51914 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0503 19:25:33.250062 51914 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0503 19:25:33.250131 51914 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0503 19:25:33.250167 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.250194 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.250231 51914 dns.go:238] Pre-creating DNS records I0503 19:25:33.250249 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.253493 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:33.255178 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.255238 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.293772 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.369495 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.370376 51914 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0503 19:25:33.370681 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:33.370778 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:33.370802 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:33.370832 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.370912 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.370866 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.370970 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:33.371045 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.372533 51914 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0503 19:25:33.372654 51914 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0503 19:25:33.372623 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0503 19:25:33.372738 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.372829 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:33.372885 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0503 19:25:33.372910 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.372947 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0503 19:25:33.373040 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.373076 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.373151 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.375440 51914 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0503 19:25:33.375526 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.375666 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.375763 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.375910 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.376032 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.376190 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.376309 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.377384 51914 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0503 19:25:33.377507 51914 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0503 19:25:33.377589 51914 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0503 19:25:33.379202 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:33.379240 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:33.379262 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:33.379281 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:33.379300 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:33.379319 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:33.379337 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:33.379355 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:33.379376 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:33.379397 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:33.379424 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:33.379445 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:33.379465 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:33.379486 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:33.379505 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:33.379524 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:33.379544 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:33.379565 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:33.379583 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:33.379601 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:33.379619 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:33.379639 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:33.379661 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:33.380430 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:33.380448 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:33.380474 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380506 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0503 19:25:33.380539 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380573 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380596 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380620 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380646 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380668 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0503 19:25:33.380694 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380728 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0503 19:25:33.380750 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0503 19:25:33.380776 51914 api.go:53] PageSize not implemented I0503 19:25:33.380789 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:33.380804 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:33.380815 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:33.380828 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:33.380840 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0503 19:25:33.380873 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.380893 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.380919 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:33.380945 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.380970 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:33.380994 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:33.381010 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.381249 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0503 19:25:33.381290 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok autoscaling-group:nodes.sharedvpc.example.com ok I0503 19:25:33.381311 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0503 19:25:33.381296 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0503 19:25:33.381334 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } volume:vol-1 ok iam-instance-profile:masters.sharedvpc.example.com ok I0503 19:25:33.381358 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:33.381388 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0503 19:25:33.381361 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0503 19:25:33.381418 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0503 19:25:33.381544 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:33.381557 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:33.381576 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0503 19:25:33.381635 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.381664 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0503 19:25:33.381729 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:33.381736 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:33.381776 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0503 19:25:33.381803 51914 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0503 19:25:33.381811 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0503 19:25:33.381830 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0503 19:25:33.381814 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:33.381855 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0503 19:25:33.381899 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0503 19:25:33.381908 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0503 19:25:33.381930 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0503 19:25:33.381971 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:33.382146 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.32s) === RUN TestLifecycleComplex I0503 19:25:33.382301 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:33.382390 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:33.382418 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:33.382444 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:33.382471 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:33.382494 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:33.382523 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:33.382555 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:33.382571 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:33.382607 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0503 19:25:33.406056 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:33.408892 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.408969 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.448730 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.583768 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.585002 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0503 19:25:33.585308 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0503 19:25:33.586450 51914 api.go:53] PageSize not implemented I0503 19:25:33.586735 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:33.586879 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:33.586875 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.586992 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:33.587090 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:33.587194 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:33.587403 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:33.587466 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0503 19:25:33.587667 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:33.587815 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:33.588141 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.588391 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.588946 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0503 19:25:33.589045 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:33.589069 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:33.589099 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:33.589140 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.589193 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0503 19:25:33.589589 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0503 19:25:33.589685 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0503 19:25:33.589760 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0503 19:25:33.589846 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0503 19:25:33.589932 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:33.590741 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:33.590824 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.590894 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:33.590923 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:33.590893 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:33.590989 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:33.590960 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:33.591023 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0503 19:25:33.591121 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0503 19:25:33.591129 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0503 19:25:33.591199 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:33.591227 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.591113 51914 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0503 19:25:33.591251 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:33.591279 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.591287 51914 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0503 19:25:33.591282 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0503 19:25:33.591368 51914 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0503 19:25:33.591396 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.591516 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0503 19:25:33.591533 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.591640 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0503 19:25:33.591760 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:33.591751 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:33.591943 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:33.592121 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.592275 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.592260 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0503 19:25:33.592360 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:33.592394 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:33.592413 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:33.592459 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.592611 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.592721 51914 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0503 19:25:33.592537 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0503 19:25:33.592753 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.592818 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.592884 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0503 19:25:33.592955 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0503 19:25:33.593037 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:33.593085 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:33.593115 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.593147 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0503 19:25:33.593173 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:33.593195 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:33.593216 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:33.593293 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:33.593320 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:33.593378 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.593407 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:33.593436 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0503 19:25:33.593490 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:33.593550 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:33.593583 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:33.593611 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:33.593639 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0503 19:25:33.594026 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:33.594095 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.594154 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.594170 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:33.594237 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:33.594223 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0503 19:25:33.594264 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0503 19:25:33.594330 51914 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0503 19:25:33.594396 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0503 19:25:33.594407 51914 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0503 19:25:33.594537 51914 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0503 19:25:33.594508 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0503 19:25:33.594583 51914 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:33.594676 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0503 19:25:33.594861 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0503 19:25:33.594976 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.595082 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0503 19:25:33.595123 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.595223 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:33.595250 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0503 19:25:33.595289 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.595344 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.595393 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.595446 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.595491 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0503 19:25:33.595529 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0503 19:25:33.595557 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:33.595604 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.595720 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0503 19:25:33.595978 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0503 19:25:33.596130 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0503 19:25:33.596226 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0503 19:25:33.596264 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" } ] }] } I0503 19:25:33.596406 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0503 19:25:33.596497 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0503 19:25:33.596633 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0503 19:25:33.596771 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I0503 19:25:33.596905 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I0503 19:25:33.597042 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0503 19:25:33.597130 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0503 19:25:33.597268 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0503 19:25:33.597418 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0503 19:25:33.597558 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0503 19:25:33.597695 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0503 19:25:33.597744 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I0503 19:25:33.597789 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0503 19:25:33.597877 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0503 19:25:33.597964 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.598058 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.598154 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0503 19:25:33.598246 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0503 19:25:33.598343 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" } ] }] } I0503 19:25:33.598499 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0503 19:25:33.598637 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0503 19:25:33.598731 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0503 19:25:33.600745 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:33.600824 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:33.600939 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:33.601022 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:33.601113 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0503 19:25:33.601152 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.601183 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.601238 51914 dns.go:238] Pre-creating DNS records I0503 19:25:33.601258 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:33.608893 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:33.612687 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.612779 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.650740 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.801125 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.802629 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0503 19:25:33.802862 51914 api.go:53] PageSize not implemented I0503 19:25:33.802800 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:33.803001 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.803105 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.803173 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.803270 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:33.803298 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:33.803351 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.803386 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.803538 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:33.803641 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:33.803732 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:33.803825 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.804712 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:33.804824 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.804924 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:33.804954 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.804995 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:33.805018 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:33.805057 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:33.805035 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.805069 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:33.805130 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:33.805157 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.805195 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.805296 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:33.805450 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.805509 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.805646 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:33.805686 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.805810 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.805891 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.806007 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:33.806139 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.806179 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:33.806307 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:33.806357 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.806436 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.806529 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:33.809532 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:33.809741 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0503 19:25:33.809762 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0503 19:25:33.809844 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:33.809885 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.809909 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:33.809998 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:33.809987 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.810035 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:33.810078 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:33.810136 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:33.810449 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:33.810179 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.810587 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.810711 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.810850 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.811345 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:33.811469 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:33.811734 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.811837 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.813277 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:33.813380 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:33.813438 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:33.813457 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:33.813475 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:33.813532 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:33.813657 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0503 19:25:33.817712 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.820119 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.826984 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:33.829339 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.829402 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:33.868469 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:33.993797 51914 iamrole.go:89] ListRoles: { } I0503 19:25:33.995020 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0503 19:25:33.995264 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:33.995390 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.995431 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:33.995509 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0503 19:25:33.995575 51914 api.go:53] PageSize not implemented I0503 19:25:33.995595 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:33.995654 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.995727 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.995792 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:33.995810 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:33.995842 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.995863 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:33.995921 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.997305 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:33.997409 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:33.997444 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:33.997535 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:33.997484 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:33.997529 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.997605 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:33.997610 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:33.997612 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.997661 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:33.997697 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:33.997732 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:33.997808 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:33.997834 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.997917 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:33.997940 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.997988 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.998061 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:33.998082 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.998127 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:33.998203 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:33.998242 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.998288 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:33.998332 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:33.998411 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:33.998434 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.000355 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:34.000506 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.000575 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.000622 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.000633 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.000626 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.000660 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.000680 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.000702 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.000720 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.000738 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.000740 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.000853 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0503 19:25:34.000835 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.000873 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.000891 51914 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0503 19:25:34.000907 51914 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0503 19:25:34.000914 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.000925 51914 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0503 19:25:34.000977 51914 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0503 19:25:34.001026 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.001045 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.001169 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.001367 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.001506 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.001716 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.001797 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.001853 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.003334 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:34.003414 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.003487 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.003508 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.003529 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.003594 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:34.003681 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0503 19:25:34.003710 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.003733 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:34.003766 51914 dns.go:238] Pre-creating DNS records I0503 19:25:34.008064 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.010374 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.010431 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.049069 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.171286 51914 iamrole.go:89] ListRoles: { } I0503 19:25:34.172576 51914 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0503 19:25:34.172886 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:34.172915 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0503 19:25:34.173029 51914 api.go:53] PageSize not implemented I0503 19:25:34.173030 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:34.173153 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:34.173244 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.173356 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:34.173385 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:34.173441 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.173624 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.173757 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.173877 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:34.173989 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.174600 51914 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0503 19:25:34.174684 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.174769 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0503 19:25:34.174875 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:34.174838 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:34.174883 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.174920 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.174980 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:34.174959 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.175006 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:34.175065 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:34.175069 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.175092 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:34.175182 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:34.175213 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.175266 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:34.175358 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:34.175390 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.175478 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:34.175559 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:34.175612 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.175762 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:34.175799 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.175924 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.176005 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.176124 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:34.177678 51914 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0503 19:25:34.177812 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.177914 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.177947 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.177998 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.177967 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.178009 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.178038 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.178059 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.178081 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.178102 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.178078 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.178120 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.178260 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.178346 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.178545 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.178698 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.178827 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.178932 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.179075 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.181718 51914 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0503 19:25:34.181790 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.181863 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.181886 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.181909 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.181972 51914 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0503 19:25:34.182059 51914 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0503 19:25:34.185397 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.188475 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.199668 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.203260 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.203375 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.241844 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.368247 51914 iamrole.go:89] ListRoles: { } I0503 19:25:34.369594 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0503 19:25:34.369852 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.369979 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.370026 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0503 19:25:34.370106 51914 api.go:53] PageSize not implemented I0503 19:25:34.370060 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:34.370218 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.370392 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:34.370504 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:34.370603 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:34.370697 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.370795 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:34.370829 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:34.370865 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:34.370968 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.371710 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:34.371813 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:34.371797 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.371813 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:34.371888 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.371890 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:34.371977 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0503 19:25:34.372043 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:34.372031 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:34.372103 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.372083 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:34.372179 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:34.372212 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.372314 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:34.372339 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.372422 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:34.372446 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:34.372479 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:34.372570 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:34.372655 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.372737 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0503 19:25:34.372832 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:34.372856 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.372909 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.372986 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.373061 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.376483 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:34.377285 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.377381 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.377441 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.377454 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.377486 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.377510 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.377432 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.377535 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.377556 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.377576 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.377583 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.377674 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.377794 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.377970 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.378101 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.378179 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.378232 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.378279 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.378404 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.379943 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.379984 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.380095 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:34.380171 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.380255 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.380278 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.380299 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.380366 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:34.380448 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:34.380482 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.380504 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:34.380539 51914 dns.go:238] Pre-creating DNS records I0503 19:25:34.384960 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.387313 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.387370 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.425619 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.554999 51914 iamrole.go:89] ListRoles: { } I0503 19:25:34.556271 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0503 19:25:34.556572 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:34.557543 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.557669 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:34.558118 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:34.558298 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:34.558419 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.558852 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.559049 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:34.559113 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:34.559117 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:34.559188 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.559301 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0503 19:25:34.559374 51914 api.go:53] PageSize not implemented I0503 19:25:34.559634 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:34.562012 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:34.562086 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.562143 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:34.562130 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:34.562180 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.562222 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:34.562172 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.562232 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:34.562322 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.562327 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:34.562420 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:34.562583 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:34.562620 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.562693 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.562793 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.562876 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.562927 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0503 19:25:34.563006 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.563053 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:34.563164 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:34.563215 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:34.563410 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:34.563457 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.563533 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.563630 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:34.563654 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:34.565411 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:34.565516 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.565604 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.565631 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.565680 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.565692 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.565716 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.565736 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.565760 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.565780 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.565756 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.565798 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.565867 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.566034 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.566106 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.566218 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.566315 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.566483 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.566608 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.566689 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.567409 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:34.567472 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.567530 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.567549 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.567565 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.567612 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:34.567676 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0503 19:25:34.570773 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.572793 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.572845 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.578124 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.580447 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.580500 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.580534 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.619796 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.744091 51914 iamrole.go:89] ListRoles: { } I0503 19:25:34.745454 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0503 19:25:34.745823 51914 api.go:53] PageSize not implemented I0503 19:25:34.745773 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.746030 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:34.746047 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.746327 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0503 19:25:34.746514 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.746640 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:34.746672 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:34.746719 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:34.746938 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:34.747115 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:34.747291 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:34.747458 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:34.747625 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.749124 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:34.749291 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:34.749502 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:34.749491 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:34.749518 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.749587 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.749667 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:34.749672 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.749747 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:34.749869 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.749844 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.749970 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:34.750032 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.750205 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.750371 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.750983 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.751296 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:34.751353 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:34.751557 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:34.751623 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.751703 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.751821 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:34.751952 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:34.751993 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.752183 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:34.752491 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:34.756084 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:34.756277 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.756452 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.756439 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.756600 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.756595 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.756639 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.756726 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.756737 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.756814 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.756883 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.756949 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.757006 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.756930 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.757102 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.757243 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.757486 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.757723 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.757863 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.757954 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.760725 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:34.760906 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.761064 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.761117 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.761175 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.761331 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:34.761656 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:34.761715 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.761752 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:34.761809 51914 dns.go:238] Pre-creating DNS records I0503 19:25:34.769747 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.773301 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.773383 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.773432 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.812373 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.938170 51914 iamrole.go:89] ListRoles: { } I0503 19:25:34.939410 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0503 19:25:34.939674 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.939773 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0503 19:25:34.939860 51914 api.go:53] PageSize not implemented I0503 19:25:34.939884 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:34.939968 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:34.939985 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:34.940122 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:34.940223 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.940338 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:34.940371 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:34.940413 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.940571 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.940695 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:34.940798 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.941247 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:34.941311 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.941375 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.941418 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:34.941395 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:34.941449 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.941442 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.941475 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:34.941529 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.941574 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:34.941520 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:34.941672 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:34.941736 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:34.941773 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:34.941884 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:34.941914 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.942015 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:34.942041 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.942109 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:34.942219 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:34.942273 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.942397 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.942465 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.942558 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:34.942617 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:34.942701 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:34.944922 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:34.945044 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:34.945107 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:34.945073 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.945124 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.945160 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.945195 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.945179 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.945233 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:34.945259 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:34.945277 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:34.945309 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.945436 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.945485 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.945545 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.945648 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.946054 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.946181 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.946383 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:34.946473 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:34.947745 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:34.947820 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:34.947898 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:34.947920 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:34.947940 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:34.947983 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:34.948088 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0503 19:25:34.949974 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:34.951985 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.952044 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.952073 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.957706 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:34.960081 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.960143 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.960176 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.960205 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:34.999534 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.123675 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.124987 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0503 19:25:35.125150 51914 api.go:53] PageSize not implemented I0503 19:25:35.125258 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0503 19:25:35.125323 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:35.125426 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:35.125549 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0503 19:25:35.125648 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:35.125720 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.125804 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:35.125861 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.125923 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:35.125941 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:35.125972 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.126000 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.126171 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:35.127105 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:35.127207 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.127330 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.127354 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:35.127405 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:35.127283 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:35.127415 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.127454 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:35.127490 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:35.127512 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:35.127505 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.127628 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:35.127778 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.127828 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.127975 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:35.128016 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.128099 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.128218 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:35.128355 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.128406 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0503 19:25:35.128533 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:35.128570 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:35.128608 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.128687 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:35.128800 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.128880 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:35.130673 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:35.130828 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.130871 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:35.130943 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:35.130927 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.130959 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.131004 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.131023 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.131040 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:35.131079 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:35.131114 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:35.131143 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:35.131201 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.131313 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.131377 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.131494 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.131589 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.131687 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.131911 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.132064 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.133079 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:35.133162 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:35.133225 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:35.133252 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:35.133278 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.133351 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:35.133464 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:35.133506 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.133537 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.133592 51914 dns.go:238] Pre-creating DNS records I0503 19:25:35.142063 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0503 19:25:35.146153 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.146238 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.146299 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.146352 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.184996 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.315805 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.317196 51914 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0503 19:25:35.317430 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.317558 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0503 19:25:35.317637 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0503 19:25:35.317737 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:35.317756 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0503 19:25:35.317802 51914 api.go:53] PageSize not implemented I0503 19:25:35.317850 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0503 19:25:35.317938 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0503 19:25:35.318035 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.318187 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0503 19:25:35.318288 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:35.318316 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:35.318377 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.318590 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.319505 51914 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0503 19:25:35.319678 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:35.319706 51914 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:35.319624 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0503 19:25:35.319676 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0503 19:25:35.319802 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:35.319754 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0503 19:25:35.319855 51914 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0503 19:25:35.319949 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.320044 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.320616 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:35.320671 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.319933 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:35.320735 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.320940 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.321047 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.321130 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0503 19:25:35.321277 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.321423 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.321579 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:35.321620 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.321750 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0503 19:25:35.321794 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:35.322066 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0503 19:25:35.322227 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.322290 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.325431 51914 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0503 19:25:35.325543 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.325670 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:35.325727 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:35.325739 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.325768 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.325735 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.325791 51914 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:35.325893 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0503 19:25:35.325916 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0503 19:25:35.325935 51914 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0503 19:25:35.326089 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.326264 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.326434 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.326755 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:35.326903 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.327040 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.327156 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.327251 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.327370 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.329319 51914 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0503 19:25:35.329429 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:35.329506 51914 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0503 19:25:35.329527 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0503 19:25:35.329550 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.329648 51914 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0503 19:25:35.329781 51914 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0503 19:25:35.330036 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:35.330091 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:35.330134 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:35.330175 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:35.330220 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:35.330264 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:35.330304 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:35.330340 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:35.330376 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:35.330415 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:35.330458 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:35.330503 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:35.330539 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:35.330572 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:35.330611 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:35.330645 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:35.330682 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:35.330721 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:35.330761 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:35.330805 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:35.330849 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:35.330886 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:35.330932 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0503 19:25:35.330976 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:35.331012 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:35.331047 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:35.331088 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:35.331128 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:35.331169 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:35.333733 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:35.333766 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:35.333808 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.333896 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0503 19:25:35.333990 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334079 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334155 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334237 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334277 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334321 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0503 19:25:35.334374 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334445 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0503 19:25:35.334490 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0503 19:25:35.334539 51914 api.go:53] PageSize not implemented I0503 19:25:35.334558 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:35.334581 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:35.334594 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0503 19:25:35.334631 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:35.334654 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:35.334670 51914 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" ] } I0503 19:25:35.334734 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.334760 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.334797 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:35.334830 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.334867 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:35.334912 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:35.334953 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0503 19:25:35.335483 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:35.335543 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I0503 19:25:35.335524 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok subnet:subnet-3 ok I0503 19:25:35.335614 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:35.335604 51914 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0503 19:25:35.335676 51914 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok autoscaling-group:nodes.complex.example.com ok I0503 19:25:35.335631 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } autoscaling-config:lt-1 ok I0503 19:25:35.335561 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:masters.complex.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok I0503 19:25:35.335664 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:35.335770 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0503 19:25:35.335802 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:35.335820 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0503 19:25:35.335760 51914 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0503 19:25:35.335829 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0503 19:25:35.336203 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.336223 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0503 19:25:35.336254 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.336302 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:35.336330 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.336374 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0503 19:25:35.336581 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.336601 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0503 19:25:35.336638 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:35.336689 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0503 19:25:35.336778 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0503 19:25:35.336823 51914 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0503 19:25:35.336796 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:35.336839 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0503 19:25:35.336869 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0503 19:25:35.336870 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0503 19:25:35.336898 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0503 19:25:35.336936 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0503 19:25:35.336948 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0503 19:25:35.336973 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0503 19:25:35.337028 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0503 19:25:35.337099 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:35.337166 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:35.337423 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.96s) === RUN TestLifecycleExternalLB I0503 19:25:35.337582 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:35.337709 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:35.337746 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:35.337784 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:35.337831 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:35.337870 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:35.337898 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:35.337935 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:35.337958 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:35.338011 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0503 19:25:35.362509 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:35.364667 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.364737 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.404914 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.479199 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.480674 51914 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0503 19:25:35.480955 51914 api.go:53] PageSize not implemented I0503 19:25:35.480903 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.481020 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.481003 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.481105 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.481082 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0503 19:25:35.481182 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:35.481210 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:35.481287 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0503 19:25:35.481321 51914 api.go:53] PageSize not implemented I0503 19:25:35.481226 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.481289 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0503 19:25:35.481375 51914 api.go:53] PageSize not implemented I0503 19:25:35.481362 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0503 19:25:35.481364 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0503 19:25:35.481431 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0503 19:25:35.481388 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.481456 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0503 19:25:35.481505 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.481567 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.481555 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.481605 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0503 19:25:35.481593 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.481673 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0503 19:25:35.481745 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.481938 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.482137 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:35.482331 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:35.482387 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:35.482429 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:35.482460 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:35.482499 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:35.482609 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:35.482648 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.483015 51914 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0503 19:25:35.483100 51914 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0503 19:25:35.483106 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.483183 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0503 19:25:35.483206 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:35.483244 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0503 19:25:35.483316 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.483319 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:35.483387 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:35.483419 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0503 19:25:35.483476 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:35.483485 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0503 19:25:35.483518 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0503 19:25:35.483657 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }], VpcId: "vpc-1" } I0503 19:25:35.483718 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:35.483749 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.483683 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0503 19:25:35.483774 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.483882 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.483928 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:35.483953 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0503 19:25:35.484063 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:35.484119 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:35.484143 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.484165 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.484277 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:35.484302 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.484417 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:35.484495 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:35.484521 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:35.485209 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0503 19:25:35.485957 51914 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0503 19:25:35.486094 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.486256 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.486354 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:35.486381 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:35.486455 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.486594 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.486677 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.486767 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:35.486801 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.486862 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.487032 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.487141 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.487286 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.487389 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.487464 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.487608 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:35.487661 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.487806 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.487935 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.488061 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.488148 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.488190 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.488340 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.488528 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0503 19:25:35.488682 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0503 19:25:35.489048 51914 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0503 19:25:35.489641 51914 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0503 19:25:35.489748 51914 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0503 19:25:35.489787 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.489818 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.489867 51914 dns.go:238] Pre-creating DNS records I0503 19:25:35.489894 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.495202 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:35.497799 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.497887 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.536851 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.628662 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.629653 51914 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0503 19:25:35.629972 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0503 19:25:35.630058 51914 api.go:53] PageSize not implemented I0503 19:25:35.630066 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.630088 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0503 19:25:35.630157 51914 api.go:53] PageSize not implemented I0503 19:25:35.630068 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.630139 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0503 19:25:35.630280 51914 api.go:53] PageSize not implemented I0503 19:25:35.630282 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:35.630250 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.630364 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.630448 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0503 19:25:35.630411 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.630508 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:35.630540 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:35.630511 51914 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0503 19:25:35.630578 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0503 19:25:35.630588 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.630621 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0503 19:25:35.630732 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.630788 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.631963 51914 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0503 19:25:35.632050 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0503 19:25:35.632166 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0503 19:25:35.632197 51914 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0503 19:25:35.632196 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.632324 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0503 19:25:35.632424 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.632461 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.632511 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:35.632562 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.632603 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.632686 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:35.632711 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.634845 51914 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0503 19:25:35.634940 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.635158 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.635296 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.635393 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.635505 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.635631 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.635732 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.637082 51914 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0503 19:25:35.637277 51914 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0503 19:25:35.637365 51914 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0503 19:25:35.639182 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:35.639222 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:35.639244 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:35.639265 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:35.639284 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:35.639307 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:35.639327 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:35.639343 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:35.639362 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:35.639381 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:35.639401 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:35.639427 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:35.639449 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:35.639467 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:35.639487 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:35.639509 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:35.639527 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:35.639546 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:35.639567 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:35.639588 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:35.639605 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:35.639624 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:35.639642 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:35.639661 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:35.639683 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:35.639704 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:35.640477 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:35.640494 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:35.640518 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640547 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0503 19:25:35.640578 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640617 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640645 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640677 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640701 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640723 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0503 19:25:35.640749 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640779 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0503 19:25:35.640801 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0503 19:25:35.640830 51914 api.go:53] PageSize not implemented I0503 19:25:35.640844 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:35.640857 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:35.640868 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:35.640881 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:35.640892 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:35.640922 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.640939 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.640969 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:35.640994 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.641023 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:35.641047 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:35.641066 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:35.641331 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:35.641403 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.641397 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0503 19:25:35.641476 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok autoscaling-group:nodes.externallb.example.com ok I0503 19:25:35.641507 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0503 19:25:35.641460 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0503 19:25:35.641584 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.641557 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0503 19:25:35.641598 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } iam-instance-profile:nodes.externallb.example.com ok security-group:sg-2 ok I0503 19:25:35.641419 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok autoscaling-config:lt-2 ok I0503 19:25:35.641644 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0503 19:25:35.641699 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.641730 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0503 19:25:35.641850 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.641861 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:35.641880 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:35.641913 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:35.641939 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0503 19:25:35.641985 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:35.642024 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0503 19:25:35.642085 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0503 19:25:35.642103 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0503 19:25:35.642137 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0503 19:25:35.642173 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0503 19:25:35.642216 51914 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0503 19:25:35.642229 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0503 19:25:35.642265 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0503 19:25:35.642337 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:35.642418 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:35.642501 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:35.642751 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.31s) === RUN TestLifecycleSharedSubnet I0503 19:25:35.642949 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:35.643068 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:35.643109 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:35.643152 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:35.643200 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:35.643239 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:35.643278 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:35.643318 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:35.643347 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:35.643403 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0503 19:25:35.666244 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:35.669378 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.669472 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.721240 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.829107 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.829935 51914 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0503 19:25:35.830146 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:35.830240 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.830320 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0503 19:25:35.830420 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0503 19:25:35.830471 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:35.830610 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:35.830669 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.830920 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.830877 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:35.830990 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" }] } I0503 19:25:35.831023 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:35.831084 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.831172 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.831272 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:35.831350 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:35.831371 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:35.831403 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0503 19:25:35.837480 51914 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0503 19:25:35.837652 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0503 19:25:35.837604 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0503 19:25:35.837722 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0503 19:25:35.837755 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0503 19:25:35.837889 51914 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0503 19:25:35.837942 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.837968 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:35.838071 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:35.838125 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.838160 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.838159 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0503 19:25:35.838359 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:35.838448 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:35.838489 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.838528 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:35.838630 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:35.838676 51914 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0503 19:25:35.839621 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0503 19:25:35.841948 51914 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0503 19:25:35.842139 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" }] }] } I0503 19:25:35.842405 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.842606 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0503 19:25:35.842741 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0503 19:25:35.842882 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.842980 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.843119 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I0503 19:25:35.843348 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.843479 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.843619 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.843830 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.844021 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.844141 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.844371 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:35.844531 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0503 19:25:35.844668 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" }] }] } I0503 19:25:35.844798 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0503 19:25:35.844922 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0503 19:25:35.846384 51914 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0503 19:25:35.846583 51914 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0503 19:25:35.846665 51914 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0503 19:25:35.846703 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.846734 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.846783 51914 dns.go:238] Pre-creating DNS records I0503 19:25:35.846808 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.852041 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:35.854590 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.854673 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.897014 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.969779 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.970638 51914 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0503 19:25:35.970888 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.971095 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:35.971687 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:35.971797 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:35.971823 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:35.971870 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.971938 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:35.972111 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.976562 51914 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0503 19:25:35.976664 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0503 19:25:35.976945 51914 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0503 19:25:35.976993 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0503 19:25:35.977049 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0503 19:25:35.977194 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.977315 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:35.977579 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0503 19:25:35.977729 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:35.981398 51914 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0503 19:25:35.981589 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.981687 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.981755 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.981933 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:35.984459 51914 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0503 19:25:35.984677 51914 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0503 19:25:35.984792 51914 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0503 19:25:35.987587 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:35.987642 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:35.987680 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:35.987715 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:35.987750 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:35.987786 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:35.987820 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:35.987855 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:35.987894 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:35.987934 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:35.987977 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:35.988013 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:35.988048 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:35.988082 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:35.988120 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:35.988157 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:35.988197 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:35.988233 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:35.988267 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:35.988308 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:35.988346 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:35.990265 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:35.990290 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:35.990332 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990383 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0503 19:25:35.990441 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990500 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990543 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990589 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990631 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990668 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0503 19:25:35.990704 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990747 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0503 19:25:35.990785 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0503 19:25:35.990832 51914 api.go:53] PageSize not implemented I0503 19:25:35.990855 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:35.990885 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:35.990906 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:35.990934 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:35.990955 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:35.991008 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:35.991038 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:35.991086 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:35.991122 51914 iamrole.go:89] ListRoles: { } I0503 19:25:35.991168 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:35.991214 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:35.991623 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0503 19:25:35.991664 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.991697 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.sharedsubnet.example.com ok I0503 19:25:35.991756 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0503 19:25:35.991839 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0503 19:25:35.991903 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0503 19:25:35.991792 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0503 19:25:35.991941 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.991957 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0503 19:25:35.991745 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I0503 19:25:35.991994 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:35.992026 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0503 19:25:35.992090 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:35.992118 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:35.992181 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0503 19:25:35.992232 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:35.992308 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0503 19:25:35.992366 51914 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0503 19:25:35.992382 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0503 19:25:35.992418 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.992481 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0503 19:25:35.992492 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0503 19:25:35.992515 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0503 19:25:35.992538 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0503 19:25:35.992706 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.35s) === RUN TestLifecyclePrivateSharedSubnet I0503 19:25:35.992843 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:35.992934 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:35.992963 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:35.992990 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:35.993020 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:35.993046 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:35.993063 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:35.993086 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:35.993102 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:35.993146 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0503 19:25:36.012053 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:36.014758 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.014836 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.014900 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.054503 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.132095 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.133342 51914 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0503 19:25:36.133696 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:36.133814 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:36.133850 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:36.133912 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.133881 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.134005 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.134091 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0503 19:25:36.134069 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.134180 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.134251 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.134270 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0503 19:25:36.134340 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.134372 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.134350 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.134403 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.134415 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0503 19:25:36.134477 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.134646 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.134698 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.135557 51914 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0503 19:25:36.135656 51914 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0503 19:25:36.135649 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0503 19:25:36.135762 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0503 19:25:36.135704 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0503 19:25:36.135950 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.135934 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0503 19:25:36.136037 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.136146 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:36.136250 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0503 19:25:36.136372 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.136436 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.136468 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.136539 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0503 19:25:36.136527 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.136579 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.136616 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0503 19:25:36.136745 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.136794 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0503 19:25:36.136972 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.137057 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0503 19:25:36.137110 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:36.137138 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.138089 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.138211 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.138315 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.138364 51914 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] } I0503 19:25:36.138482 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:36.138523 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:36.138586 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:36.138627 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:36.138682 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.138746 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.138804 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:36.138853 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.138905 51914 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] } I0503 19:25:36.139559 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.141292 51914 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0503 19:25:36.141648 51914 api.go:53] PageSize not implemented I0503 19:25:36.141533 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } W0503 19:25:36.141735 51914 api.go:53] PageSize not implemented I0503 19:25:36.141721 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.141821 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0503 19:25:36.141917 51914 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0503 19:25:36.141903 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.141970 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0503 19:25:36.142004 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0503 19:25:36.142048 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0503 19:25:36.142031 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0503 19:25:36.142114 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0503 19:25:36.142168 51914 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0503 19:25:36.142212 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0503 19:25:36.142161 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0503 19:25:36.142242 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0503 19:25:36.142300 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0503 19:25:36.142273 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.142401 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.142545 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.142662 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:36.142815 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.142914 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.143052 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.143118 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.143181 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.143289 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:36.143375 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:36.143467 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0503 19:25:36.143560 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0503 19:25:36.143618 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.143713 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0503 19:25:36.143822 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.143938 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.144046 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.144166 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.144287 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.144400 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0503 19:25:36.144483 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0503 19:25:36.144559 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.144626 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.144705 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0503 19:25:36.146214 51914 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0503 19:25:36.146305 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.146464 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.146577 51914 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0503 19:25:36.146709 51914 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0503 19:25:36.146814 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.146887 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.146964 51914 dns.go:238] Pre-creating DNS records I0503 19:25:36.147027 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.150930 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:36.152914 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.152976 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.153012 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.192012 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.268321 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.269655 51914 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0503 19:25:36.269863 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.272290 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.272409 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:36.272549 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:36.272879 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.272993 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.273124 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:36.273322 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.277567 51914 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0503 19:25:36.277759 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0503 19:25:36.277776 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0503 19:25:36.277903 51914 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0503 19:25:36.277958 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0503 19:25:36.277925 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:36.278063 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0503 19:25:36.278183 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.278222 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0503 19:25:36.278293 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0503 19:25:36.278382 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.278414 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0503 19:25:36.278495 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.278545 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0503 19:25:36.278630 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:36.278679 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0503 19:25:36.278804 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.281362 51914 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0503 19:25:36.281563 51914 api.go:53] PageSize not implemented I0503 19:25:36.281593 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0503 19:25:36.281563 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.281715 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0503 19:25:36.281768 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0503 19:25:36.281777 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0503 19:25:36.281865 51914 api.go:53] PageSize not implemented I0503 19:25:36.281885 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0503 19:25:36.281942 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0503 19:25:36.281965 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0503 19:25:36.282189 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.282368 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.282685 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.282843 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.284334 51914 executor.go:111] Tasks: 100 done / 107 total; 4 can run W0503 19:25:36.284415 51914 api.go:53] PageSize not implemented I0503 19:25:36.284411 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.284467 51914 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0503 19:25:36.284435 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } W0503 19:25:36.284588 51914 api.go:53] PageSize not implemented I0503 19:25:36.284608 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0503 19:25:36.284690 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0503 19:25:36.284755 51914 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0503 19:25:36.284878 51914 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0503 19:25:36.287885 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:36.287943 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:36.287975 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:36.288005 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:36.288036 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.288066 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:36.288095 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:36.288133 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:36.288167 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:36.288199 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:36.288233 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0503 19:25:36.288272 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:36.288304 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:36.288334 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:36.288363 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.288393 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.288426 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:36.288456 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:36.288490 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:36.288524 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:36.288558 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:36.288591 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:36.288620 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.288655 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.288691 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:36.290172 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:36.290196 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:36.290235 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290295 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0503 19:25:36.290364 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290422 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290462 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290500 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290541 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290577 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0503 19:25:36.290612 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290654 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0503 19:25:36.290689 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0503 19:25:36.290731 51914 api.go:53] PageSize not implemented I0503 19:25:36.290750 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0503 19:25:36.290797 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:36.290820 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:36.290836 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:36.290857 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:36.290873 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:36.290920 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.290950 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.290994 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:36.291035 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.291075 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:36.291113 51914 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0503 19:25:36.291603 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0503 19:25:36.291609 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } autoscaling-group:nodes.private-shared-subnet.example.com ok I0503 19:25:36.291631 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } load-balancer:api-private-shared-subnet-n2f8ak ok I0503 19:25:36.291688 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.291682 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0503 19:25:36.291674 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:bastions.private-shared-subnet.example.com ok volume:vol-2 ok I0503 19:25:36.291724 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.291747 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:bastion.private-shared-subnet.example.com ok I0503 19:25:36.291772 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0503 19:25:36.291723 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0503 19:25:36.291832 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.291803 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0503 19:25:36.291877 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok W0503 19:25:36.291885 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.291905 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I0503 19:25:36.291928 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0503 19:25:36.291968 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.291999 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0503 19:25:36.292129 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.292142 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0503 19:25:36.292172 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.292198 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0503 19:25:36.292281 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.292289 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:36.292330 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.292351 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.292367 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0503 19:25:36.292414 51914 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com W0503 19:25:36.292418 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.292431 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.292470 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0503 19:25:36.292479 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.292499 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } I0503 19:25:36.292445 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } iam-role:nodes.private-shared-subnet.example.com ok security-group:sg-4 ok I0503 19:25:36.292527 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.292533 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.292570 51914 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0503 19:25:36.292584 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0503 19:25:36.292614 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0503 19:25:36.292645 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0503 19:25:36.292576 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0503 19:25:36.292679 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.292695 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0503 19:25:36.292674 51914 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0503 19:25:36.292885 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.30s) === RUN TestLifecyclePrivateSharedIP I0503 19:25:36.293033 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:36.293133 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:36.293160 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:36.293190 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:36.293221 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:36.293246 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:36.293263 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:36.293284 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:36.293300 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:36.293334 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0503 19:25:36.310397 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:36.312315 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.312382 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.312417 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.353533 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.444338 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.445558 51914 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0503 19:25:36.445820 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.445823 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.445888 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.445917 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0503 19:25:36.446049 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:36.446079 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0503 19:25:36.446138 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:36.446177 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:36.446150 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0503 19:25:36.446137 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.446214 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.446306 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.446386 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.446473 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0503 19:25:36.446530 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.446675 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.446785 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.446830 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.446908 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.447719 51914 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0503 19:25:36.447819 51914 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0503 19:25:36.447825 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.447915 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.447945 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.448712 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0503 19:25:36.448764 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.449022 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.448022 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I0503 19:25:36.449111 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0503 19:25:36.448028 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0503 19:25:36.449214 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I0503 19:25:36.449395 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:36.449484 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0503 19:25:36.449524 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0503 19:25:36.449752 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.449866 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:36.449820 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.450025 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0503 19:25:36.450026 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0503 19:25:36.450326 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.450440 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.450573 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0503 19:25:36.450849 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0503 19:25:36.451214 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.451314 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.451435 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.451525 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.451569 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:36.451647 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0503 19:25:36.452179 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0503 19:25:36.452277 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.452356 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.452327 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0503 19:25:36.452405 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.452461 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:36.452499 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:36.452547 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:36.452579 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.452411 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0503 19:25:36.452662 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:36.452722 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:36.452771 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0503 19:25:36.452904 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:36.452977 51914 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0503 19:25:36.453093 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:36.453155 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.453210 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.453266 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:36.454045 51914 executor.go:111] Tasks: 71 done / 116 total; 36 can run W0503 19:25:36.454174 51914 api.go:53] PageSize not implemented W0503 19:25:36.454292 51914 api.go:53] PageSize not implemented I0503 19:25:36.454201 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:36.454334 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0503 19:25:36.454400 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.454431 51914 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0503 19:25:36.454504 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.454561 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0503 19:25:36.454601 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0503 19:25:36.454636 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" }] }] } I0503 19:25:36.454656 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0503 19:25:36.454720 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0503 19:25:36.454779 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0503 19:25:36.454832 51914 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0503 19:25:36.454873 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0503 19:25:36.454835 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0503 19:25:36.454891 51914 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0503 19:25:36.454922 51914 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0503 19:25:36.454967 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0503 19:25:36.455030 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0503 19:25:36.455086 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.455160 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0503 19:25:36.455303 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.455451 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0503 19:25:36.455512 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.455664 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.455734 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:36.455838 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0503 19:25:36.455955 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.456140 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I0503 19:25:36.456298 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.456455 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.456713 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" }] }] } I0503 19:25:36.456860 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:36.456965 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.457009 51914 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0503 19:25:36.457054 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.457180 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.457322 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0503 19:25:36.457431 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.457513 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.457584 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0503 19:25:36.457629 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0503 19:25:36.457760 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0503 19:25:36.457884 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0503 19:25:36.457973 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0503 19:25:36.458100 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.458174 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0503 19:25:36.458298 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.458412 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.458487 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0503 19:25:36.458526 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.458647 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.458765 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.458883 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:36.458946 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:36.460224 51914 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0503 19:25:36.460295 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.460311 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:36.460444 51914 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.460442 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.460527 51914 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0503 19:25:36.460581 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:36.460606 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.460631 51914 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0503 19:25:36.460668 51914 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0503 19:25:36.460724 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:36.460849 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0503 19:25:36.460906 51914 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0503 19:25:36.460941 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.460970 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.461013 51914 dns.go:238] Pre-creating DNS records I0503 19:25:36.461033 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.466933 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0503 19:25:36.469800 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.469886 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.469942 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.508637 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.618168 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.619501 51914 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0503 19:25:36.619733 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.619803 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.619936 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0503 19:25:36.619959 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.620055 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0503 19:25:36.620092 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0503 19:25:36.620129 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] }] } I0503 19:25:36.620307 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.621584 51914 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0503 19:25:36.621695 51914 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0503 19:25:36.621745 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0503 19:25:36.621681 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0503 19:25:36.621871 51914 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0503 19:25:36.621838 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.622002 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0503 19:25:36.622155 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.622209 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0503 19:25:36.622333 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0503 19:25:36.622458 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.622503 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0503 19:25:36.622598 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0503 19:25:36.622691 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.622723 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:36.622757 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.622839 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:36.622862 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.622934 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0503 19:25:36.622958 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0503 19:25:36.623041 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.624547 51914 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0503 19:25:36.624637 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0503 19:25:36.624711 51914 api.go:53] PageSize not implemented I0503 19:25:36.624718 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.624729 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0503 19:25:36.624786 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0503 19:25:36.624773 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W0503 19:25:36.624814 51914 api.go:53] PageSize not implemented I0503 19:25:36.624821 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.624843 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0503 19:25:36.624906 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0503 19:25:36.624931 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0503 19:25:36.624953 51914 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0503 19:25:36.624955 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.625123 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.625484 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.625691 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.625804 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.625959 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.626059 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.626116 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0503 19:25:36.627437 51914 executor.go:111] Tasks: 107 done / 116 total; 5 can run W0503 19:25:36.627505 51914 api.go:53] PageSize not implemented I0503 19:25:36.627518 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0503 19:25:36.627553 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:36.627614 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:36.627576 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0503 19:25:36.627651 51914 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" W0503 19:25:36.627623 51914 api.go:53] PageSize not implemented I0503 19:25:36.627685 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0503 19:25:36.627847 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0503 19:25:36.627921 51914 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0503 19:25:36.627994 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0503 19:25:36.628095 51914 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0503 19:25:36.631099 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:36.631156 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:36.631191 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:36.631223 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:36.631254 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.631286 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:36.631317 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:36.631347 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:36.631380 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:36.631415 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:36.631450 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0503 19:25:36.631492 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0503 19:25:36.631527 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:36.631556 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:36.631586 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:36.631619 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:36.631650 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0503 19:25:36.631682 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.631713 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.631744 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0503 19:25:36.631777 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0503 19:25:36.631809 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0503 19:25:36.631839 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.631873 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:36.631902 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0503 19:25:36.631935 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:36.631962 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:36.631990 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.632023 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.632054 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:36.633539 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:36.633566 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:36.633604 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633665 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0503 19:25:36.633737 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633791 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633829 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633868 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633903 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.633947 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0503 19:25:36.633993 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.634041 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:36.634067 51914 natgateway.go:117] DescribeNatGateways: { } I0503 19:25:36.634106 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0503 19:25:36.634140 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0503 19:25:36.634184 51914 api.go:53] PageSize not implemented I0503 19:25:36.634203 51914 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0503 19:25:36.634253 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:36.634279 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:36.634295 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:36.634317 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:36.634333 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:36.634377 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.634404 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.634444 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:36.634480 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.634527 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:36.634572 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:36.634605 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0503 19:25:36.634638 51914 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0503 19:25:36.634668 51914 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:nodes.private-shared-ip.example.com ok autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok autoscaling-group:bastion.private-shared-ip.example.com ok I0503 19:25:36.635291 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0503 19:25:36.635320 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.635379 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0503 19:25:36.635367 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0503 19:25:36.635424 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.635426 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0503 19:25:36.635475 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0503 19:25:36.635526 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0503 19:25:36.635558 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0503 19:25:36.635486 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0503 19:25:36.635734 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.635749 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0503 19:25:36.635623 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok autoscaling-config:lt-2 ok I0503 19:25:36.635861 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0503 19:25:36.635932 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0503 19:25:36.635910 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0503 19:25:36.635998 51914 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok W0503 19:25:36.636020 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-3 ok I0503 19:25:36.636047 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0503 19:25:36.636075 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0503 19:25:36.636125 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0503 19:25:36.636199 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.636212 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0503 19:25:36.636254 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:36.636313 51914 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0503 19:25:36.636377 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0503 19:25:36.636393 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.636446 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0503 19:25:36.636422 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:36.636461 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0503 19:25:36.636477 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.636499 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0503 19:25:36.636513 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0503 19:25:36.636568 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.636576 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:36.636552 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.636595 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.636652 51914 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0503 19:25:36.636665 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0503 19:25:36.636688 51914 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0503 19:25:36.636714 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0503 19:25:36.636740 51914 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0503 19:25:36.636666 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0503 19:25:36.636747 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } W0503 19:25:36.636759 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.636763 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0503 19:25:36.636770 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:36.636846 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:36.636900 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0503 19:25:36.637123 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.34s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0503 19:25:36.637277 51914 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:36.637369 51914 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:36.637404 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:36.637432 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:36.637463 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:36.637489 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:36.637507 51914 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:36.637531 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:36.637547 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:36.637582 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0503 19:25:36.652358 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0503 19:25:36.654135 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.654198 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.693602 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.802389 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.803372 51914 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0503 19:25:36.803546 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.803594 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.803654 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0503 19:25:36.803716 51914 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.803744 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] } I0503 19:25:36.803820 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.803845 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.803930 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.804027 51914 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.804114 51914 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0503 19:25:36.804150 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.804355 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.804549 51914 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.804549 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.804702 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.804757 51914 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.804862 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:36.804904 51914 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0503 19:25:36.805016 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.805070 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0503 19:25:36.805104 51914 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0503 19:25:36.805130 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:36.805166 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.806270 51914 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0503 19:25:36.806635 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0503 19:25:36.806882 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.807013 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0503 19:25:36.807005 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0503 19:25:36.807204 51914 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0503 19:25:36.807161 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.807345 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.807373 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:36.807355 51914 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0503 19:25:36.807755 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.808050 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0503 19:25:36.808191 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.808233 51914 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0503 19:25:36.808258 51914 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0503 19:25:36.808309 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.808369 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.808393 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0503 19:25:36.808541 51914 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:36.808598 51914 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0503 19:25:36.808647 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:36.809225 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.809436 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.809402 51914 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I0503 19:25:36.809500 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.809531 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:36.809557 51914 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" } ] } I0503 19:25:36.809614 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.809724 51914 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.809774 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:36.809715 51914 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.811795 51914 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0503 19:25:36.811911 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.812027 51914 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:36.812078 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.812219 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0503 19:25:36.812273 51914 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0503 19:25:36.812308 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.812427 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.812585 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0503 19:25:36.812688 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.812767 51914 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0503 19:25:36.812854 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0503 19:25:36.813010 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.813119 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.813276 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.813353 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.813493 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.813658 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.813749 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.813861 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.813966 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.814068 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.814273 51914 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0503 19:25:36.814425 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0503 19:25:36.814568 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0503 19:25:36.814710 51914 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0503 19:25:36.815056 51914 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0503 19:25:36.815209 51914 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0503 19:25:36.815390 51914 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0503 19:25:36.815429 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.815455 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.815495 51914 dns.go:238] Pre-creating DNS records I0503 19:25:36.815514 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.819014 51914 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0503 19:25:36.820800 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.820862 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.860608 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.977483 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.978441 51914 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0503 19:25:36.978799 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.978941 51914 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0503 19:25:36.978956 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.979111 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.979170 51914 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0503 19:25:36.979230 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.979323 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:36.979343 51914 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:36.979373 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.979405 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.980744 51914 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0503 19:25:36.980840 51914 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0503 19:25:36.980862 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.980983 51914 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0503 19:25:36.981000 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.981133 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.981224 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0503 19:25:36.981386 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.981435 51914 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:36.981513 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0503 19:25:36.981632 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.981677 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.981828 51914 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:36.985039 51914 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0503 19:25:36.985268 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.985363 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.985459 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.985550 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.985692 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.985823 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.986001 51914 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0503 19:25:36.988380 51914 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0503 19:25:36.988576 51914 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0503 19:25:36.988709 51914 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0503 19:25:36.990733 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0503 19:25:36.990779 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0503 19:25:36.990817 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0503 19:25:36.990853 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0503 19:25:36.990889 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0503 19:25:36.990928 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0503 19:25:36.990961 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0503 19:25:36.990995 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0503 19:25:36.991029 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0503 19:25:36.991066 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0503 19:25:36.991100 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0503 19:25:36.991144 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0503 19:25:36.991182 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0503 19:25:36.991211 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0503 19:25:36.991244 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0503 19:25:36.991281 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0503 19:25:36.991313 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:36.991347 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0503 19:25:36.991381 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:36.991417 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0503 19:25:36.991449 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0503 19:25:36.991481 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0503 19:25:36.991514 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0503 19:25:36.991550 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0503 19:25:36.991584 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:36.991619 51914 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0503 19:25:36.992498 51914 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0503 19:25:36.992522 51914 keypairs.go:95] DescribeKeyPairs: { } I0503 19:25:36.992561 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992603 51914 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992652 51914 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992700 51914 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992742 51914 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992791 51914 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992824 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992863 51914 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992906 51914 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992951 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0503 19:25:36.992985 51914 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0503 19:25:36.993033 51914 api.go:53] PageSize not implemented I0503 19:25:36.993053 51914 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0503 19:25:36.993080 51914 loadbalancers.go:34] PageSize not implemented I0503 19:25:36.993099 51914 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0503 19:25:36.993120 51914 targetgroups.go:35] PageSize not implemented I0503 19:25:36.993138 51914 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0503 19:25:36.993181 51914 zones.go:93] ListHostedZonesPages { } I0503 19:25:36.993209 51914 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0503 19:25:36.993251 51914 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0503 19:25:36.993286 51914 iamrole.go:89] ListRoles: { } I0503 19:25:36.993326 51914 internetgateways.go:101] DescribeInternetGateways: { } I0503 19:25:36.993362 51914 routetable.go:53] DescribeRouteTables: { } I0503 19:25:36.993391 51914 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0503 19:25:36.993684 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0503 19:25:36.993692 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0503 19:25:36.993728 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0503 19:25:36.993785 51914 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.993775 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0503 19:25:36.993824 51914 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-config:lt-2 ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0503 19:25:36.993835 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0503 19:25:36.993966 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.993980 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0503 19:25:36.993802 51914 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok I0503 19:25:36.994007 51914 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0503 19:25:36.994041 51914 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0503 19:25:36.994071 51914 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0503 19:25:36.994131 51914 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0503 19:25:36.994140 51914 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0503 19:25:36.994160 51914 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0503 19:25:36.994212 51914 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0503 19:25:36.994247 51914 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0503 19:25:36.994303 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0503 19:25:36.994334 51914 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0503 19:25:36.994321 51914 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0503 19:25:36.994344 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0503 19:25:36.994390 51914 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0503 19:25:36.994426 51914 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.994471 51914 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0503 19:25:36.994486 51914 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0503 19:25:36.994518 51914 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0503 19:25:36.994588 51914 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0503 19:25:36.994667 51914 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0503 19:25:36.994742 51914 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0503 19:25:36.994983 51914 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.36s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 49.148s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.004s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0503 19:24:51.430121 52444 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.003s === RUN TestPodController I0503 19:24:51.459807 52446 pod.go:59] starting pod controller I0503 19:24:51.460045 52446 stoppable.go:62] shutting down controller I0503 19:24:51.460126 52446 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS I0503 19:24:51.460238 52446 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.036s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a7f40} route53_test.go:93: Got zone list: [0xc0000a7f70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc0000120c0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003600a0} route53_test.go:93: Got zone list: [0xc0003600d0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc0000120d8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0003601d0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003338f0 0xc0000120f8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360390} route53_test.go:93: Got zone list: [0xc0003603c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012118} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360520} route53_test.go:93: Got zone list: [0xc000360550] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012140} route53_test.go:187: Successfully added resource record set: {0xc000182600 0xc000012160} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360830} route53_test.go:93: Got zone list: [0xc000360860] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc0000121f0} route53_test.go:199: Successfully added resource record set: {0xc000182780 0xc000012210} route53_test.go:118: Got 1 recordsets: [0xc000360a00] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360b90} route53_test.go:93: Got zone list: [0xc000360bc0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc0000122b0} route53_test.go:221: Successfully added resource record set: {0xc000182900 0xc0000122d0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000182900 0xc0000122d0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360f80} route53_test.go:93: Got zone list: [0xc000360fb0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012390} route53_test.go:246: Successfully removed resource set {0xc000182b00 0xc0000123b0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361290} route53_test.go:93: Got zone list: [0xc0003612c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012440} route53_test.go:264: Successfully removed resource set {0xc000182c80 0xc000012460} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003615d0} route53_test.go:93: Got zone list: [0xc000361600] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc0000124f8} commontests.go:192: Successfully added resource record set: {0xc000182e00 0xc000012518} commontests.go:45: Correctly replaced resource record {0xc000182e00 0xc000012518} -> {0xc000182f00 0xc000012568} commontests.go:142: Got 1 recordsets: [0xc000361910] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361b40} route53_test.go:93: Got zone list: [0xc000361b70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012620} commontests.go:192: Successfully added resource record set: {0xc000183180 0xc000012640} commontests.go:70: Correctly replaced resource record {0xc000183180 0xc000012640} -> {0xc000183280 0xc000012690} commontests.go:142: Got 1 recordsets: [0xc000361e80] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036e100} route53_test.go:93: Got zone list: [0xc00036e130] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012758} commontests.go:192: Successfully added resource record set: {0xc000183500 0xc000012778} commontests.go:142: Got 2 recordsets: [0xc00036e3a0 0xc00036e3c0] commontests.go:142: Got 2 recordsets: [0xc00036e430 0xc00036e450] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036e650} route53_test.go:93: Got zone list: [0xc00036e680] route53_test.go:98: Got at least 1 zone in list:&{0xc0003336b0 0xc000012880} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.014s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.006s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625260} clouddns_test.go:71: Got zone list: [0xc0001ae408] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001ae3d8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625360} clouddns_test.go:71: Got zone list: [0xc0001ae4c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001ae498} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625460} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00062da80 0xc0001ae588} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625520} clouddns_test.go:71: Got zone list: [0xc0001ae648] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001ae618} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625640} clouddns_test.go:71: Got zone list: [0xc0001ae780] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001ae750} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625780} clouddns_test.go:71: Got zone list: [0xc0001ae8e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001ae8b8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc0006258e0} clouddns_test.go:71: Got zone list: [0xc0001aeb88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001aeb58} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625a40} clouddns_test.go:71: Got zone list: [0xc0001aed20] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001aecf0} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625b80} clouddns_test.go:71: Got zone list: [0xc0001aee88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001aee58} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625ce0} clouddns_test.go:71: Got zone list: [0xc0001af050] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001af020} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625e40} clouddns_test.go:71: Got zone list: [0xc0001af320] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001af2f0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc000625fc0} clouddns_test.go:71: Got zone list: [0xc0001af7d0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001af7a0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000120b10 0xc00018c1c0} clouddns_test.go:71: Got zone list: [0xc0001afba8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00062d900 0xc0001afb78} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.052s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.025s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.065s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0503 19:24:59.527970 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.528214 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.528289 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.528363 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.528457 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.528517 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.528553 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.528617 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.528684 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.543560 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0503 19:24:59.550010 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.550173 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.550276 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.550340 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.550409 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.550456 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.550489 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.550531 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.550560 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.552460 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0503 19:24:59.558226 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.558350 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.558396 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.558439 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.558486 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.558526 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.558556 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.558607 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.558636 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.560581 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.563871 52996 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0503 19:24:59.563905 52996 assetstore.go:111] bin/containerd I0503 19:24:59.563919 52996 assetstore.go:111] bin/containerd-shim I0503 19:24:59.563933 52996 assetstore.go:111] bin/containerd-shim-runc-v1 I0503 19:24:59.563946 52996 assetstore.go:111] bin/containerd-shim-runc-v2 I0503 19:24:59.563968 52996 assetstore.go:111] bin/containerd-stress I0503 19:24:59.563983 52996 assetstore.go:111] bin/ctr I0503 19:24:59.564050 52996 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0503 19:24:59.564071 52996 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0503 19:24:59.569069 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.569214 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.569262 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.569307 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.569356 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.569396 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.569425 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.569462 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.569488 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.571218 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.573456 52996 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.01s) === RUN TestContainerdBuilder_SkipInstall I0503 19:24:59.574955 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.575100 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.575146 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.575193 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.575243 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.575356 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.575387 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.575426 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.575453 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.577143 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.579401 52996 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0503 19:24:59.580588 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.580709 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.580750 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.580788 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.580829 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.580909 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.580938 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.580975 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.580999 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.584765 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.587372 52996 assetstore.go:108] Matching assets for "^docker/": I0503 19:24:59.587491 52996 assetstore.go:111] docker/containerd I0503 19:24:59.587537 52996 assetstore.go:111] docker/containerd-shim I0503 19:24:59.587582 52996 assetstore.go:111] docker/ctr I0503 19:24:59.587628 52996 assetstore.go:111] docker/docker I0503 19:24:59.587643 52996 assetstore.go:111] docker/docker-init I0503 19:24:59.587656 52996 assetstore.go:111] docker/docker-proxy I0503 19:24:59.587669 52996 assetstore.go:111] docker/dockerd I0503 19:24:59.587682 52996 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0503 19:24:59.590991 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.591112 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.591160 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.591205 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.591250 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.591315 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.591348 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.591387 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.591415 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.592701 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.594387 52996 assetstore.go:108] Matching assets for "^docker/": I0503 19:24:59.594409 52996 assetstore.go:111] docker/containerd I0503 19:24:59.594423 52996 assetstore.go:111] docker/containerd-shim I0503 19:24:59.594444 52996 assetstore.go:111] docker/ctr I0503 19:24:59.594459 52996 assetstore.go:111] docker/docker I0503 19:24:59.594471 52996 assetstore.go:111] docker/docker-init I0503 19:24:59.594483 52996 assetstore.go:111] docker/docker-proxy I0503 19:24:59.594495 52996 assetstore.go:111] docker/dockerd I0503 19:24:59.594508 52996 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0503 19:24:59.597676 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.597791 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.597850 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.597924 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.597996 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.598072 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.598093 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.598119 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.598136 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.599263 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.600805 52996 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0503 19:24:59.601020 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.601107 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.601137 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.601166 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.601193 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.601249 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.601269 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.601292 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.601307 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.602437 52996 zones.go:93] ListHostedZonesPages { } I0503 19:24:59.604210 52996 assetstore.go:108] Matching assets for "^docker/": I0503 19:24:59.604231 52996 assetstore.go:111] docker/docker I0503 19:24:59.604242 52996 assetstore.go:111] docker/docker-containerd I0503 19:24:59.604250 52996 assetstore.go:111] docker/docker-containerd-ctr I0503 19:24:59.604257 52996 assetstore.go:111] docker/docker-containerd-shim I0503 19:24:59.604264 52996 assetstore.go:111] docker/docker-init I0503 19:24:59.604271 52996 assetstore.go:111] docker/docker-proxy I0503 19:24:59.604278 52996 assetstore.go:111] docker/docker-runc I0503 19:24:59.604285 52996 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0503 19:24:59.608484 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.608596 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.608634 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.608661 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.608694 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.608720 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.608737 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.608763 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.608780 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.612433 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0503 19:24:59.616816 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.616946 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.616999 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.617044 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.617098 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.617144 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.617174 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.617211 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.617237 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.620559 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0503 19:24:59.624812 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.624944 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.624996 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.625045 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.625092 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.625134 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.625177 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.625223 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.625252 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.628301 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.01s) === RUN TestKubeAPIServerBuilder I0503 19:24:59.640081 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.640217 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.640265 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.640314 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.640362 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.640401 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.640429 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.640469 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.640495 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.642679 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0503 19:24:59.656858 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.656985 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.657033 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.657071 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.657115 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.657151 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.657201 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.657239 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.657262 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.660111 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0503 19:24:59.669314 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.669414 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.669454 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.669491 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.669523 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.669583 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.669604 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.669637 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.669658 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.672008 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.02s) === RUN TestAwsIamAuthenticator I0503 19:24:59.687782 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.687921 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.687974 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.688033 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.688089 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.688136 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.688167 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.688207 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.688234 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.691402 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0503 19:24:59.701071 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:24:59.701200 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:24:59.701248 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:24:59.701290 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:24:59.701339 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:24:59.701383 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:24:59.701412 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:24:59.701449 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:24:59.701473 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:24:59.704485 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.46s) === RUN TestKubeAPIServerBuilderARM64 I0503 19:25:01.164533 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.164638 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.164676 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.164707 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.164739 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.164765 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.164781 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.164803 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.164819 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.166757 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0503 19:25:01.247579 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.247713 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.247763 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.247812 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.247860 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.247900 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.247930 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.247967 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.247992 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.250975 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0503 19:25:01.261820 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.261951 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.261995 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.262035 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.262081 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.262118 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.262146 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.262182 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.262205 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.265120 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0503 19:25:01.336845 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.336943 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.336979 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.337010 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.337042 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.337067 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.337083 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.337104 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.337119 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.339074 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.09s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0503 19:25:01.422850 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.422982 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.423031 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.423082 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.423133 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.423180 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.423214 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.423254 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.423281 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.426461 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0503 19:25:01.433065 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.433198 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.433245 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.433286 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.433338 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.433380 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.433411 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.433451 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.433478 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.436701 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0503 19:25:01.505578 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.505682 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.505716 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.505747 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.505778 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.505804 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.505820 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.505843 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.505858 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.507990 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0503 19:25:01.577958 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.578057 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.578093 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.578124 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.578173 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.578219 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.578251 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.578291 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.578318 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.580250 52996 zones.go:93] ListHostedZonesPages { } W0503 19:25:01.582797 52996 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0503 19:25:01.584549 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.584658 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.584692 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.584719 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.584749 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.584773 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.584788 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.584901 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.585234 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.589280 52996 zones.go:93] ListHostedZonesPages { } W0503 19:25:01.656859 52996 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0503 19:25:01.658948 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.659058 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.659097 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.659125 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.659156 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.659180 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.659196 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.659217 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.659231 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.661137 52996 zones.go:93] ListHostedZonesPages { } W0503 19:25:01.734907 52996 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I0503 19:25:01.736659 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.736749 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.736781 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.736810 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.736841 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.736865 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.736880 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.736903 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.736919 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.738592 52996 zones.go:93] ListHostedZonesPages { } I0503 19:25:01.740700 52996 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0503 19:25:01.742553 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.742620 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.742660 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.742686 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.742712 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.742733 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.742749 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.742769 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.742782 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.744100 52996 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I0503 19:25:01.747325 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.747389 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.747429 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.747455 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.747498 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.747519 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.747536 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.747557 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.747570 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.748801 52996 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0503 19:25:01.751851 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.751922 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.751945 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.751968 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.751993 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.752016 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.752034 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.752053 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.752067 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.755467 52996 zones.go:93] ListHostedZonesPages { } I0503 19:25:01.757176 52996 assetstore.go:108] Matching assets for "protokube$": I0503 19:25:01.757196 52996 assetstore.go:111] /opt/kops/bin/protokube I0503 19:25:01.757208 52996 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0503 19:25:01.757230 52996 assetstore.go:108] Matching assets for "channels$": I0503 19:25:01.757239 52996 assetstore.go:111] /opt/kops/bin/channels I0503 19:25:01.757246 52996 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0503 19:25:01.758600 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.758688 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.758718 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.758747 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.758777 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.758805 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.758823 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.758847 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.758864 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.760652 52996 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0503 19:25:01.763005 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.763096 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.763141 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.763178 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.763209 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.763235 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.763252 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.763274 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.763290 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.765496 52996 zones.go:93] ListHostedZonesPages { } I0503 19:25:01.768006 52996 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I0503 19:25:01.768788 52996 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:01.768988 52996 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:01.769073 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:01.769152 52996 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:01.769233 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:01.769308 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:01.769377 52996 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:01.769456 52996 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:01.769519 52996 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:01.772308 52996 zones.go:93] ListHostedZonesPages { } I0503 19:25:01.774097 52996 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 2.337s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.031s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.039s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.025s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.025s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0503 19:24:59.454376 53001 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0503 19:24:59.454512 53001 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.011s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0503 19:24:59.513919 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514082 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514147 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514200 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514262 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0503 19:24:59.514676 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514714 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514742 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514799 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514878 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514925 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514958 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.514987 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515021 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515058 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515093 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515128 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515158 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515185 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515210 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515247 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515276 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515309 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0503 19:24:59.515336 53003 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0503 19:24:59.515541 53003 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0503 19:24:59.515617 53003 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.084s === RUN TestApplySet I0503 19:24:59.481648 53004 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0503 19:24:59.482727 53004 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0503 19:24:59.483877 53004 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0503 19:24:59.483992 53004 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0503 19:24:59.485462 53004 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0503 19:24:59.485857 53004 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0503 19:24:59.486651 53004 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0503 19:24:59.486654 53004 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0503 19:24:59.487430 53004 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I0503 19:24:59.487536 53004 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I0503 19:24:59.487914 53004 health.go:33] status conditions not found for /v1, Kind=Namespace I0503 19:24:59.488224 53004 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I0503 19:24:59.488315 53004 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I0503 19:24:59.489025 53004 health.go:33] status conditions not found for /v1, Kind=ConfigMap I0503 19:24:59.489392 53004 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I0503 19:24:59.489472 53004 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I0503 19:24:59.490498 53004 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I0503 19:24:59.490562 53004 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.051s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0503 19:24:59.589404 53005 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.11s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0503 19:24:59.591032 53005 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0503 19:24:59.592497 53005 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0503 19:24:59.593583 53005 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0503 19:24:59.594617 53005 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0503 19:24:59.595871 53005 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0503 19:24:59.596842 53005 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0503 19:24:59.597695 53005 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0503 19:24:59.597715 53005 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0503 19:24:59.597727 53005 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0503 19:24:59.597739 53005 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.156s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.007s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.061s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.022s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.076s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.028s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.007s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.005s === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.037s === RUN TestFlagToFalse I0503 19:25:07.127570 54452 featureflag_test.go:32] Created flag Unittest1 I0503 19:25:07.127623 54452 featureflag.go:160] FeatureFlag "UnitTest1"=false I0503 19:25:07.127633 54452 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0503 19:25:07.127712 54452 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.004s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.031s === RUN TestRenderInstanceGroupZones W0503 19:25:07.146336 54457 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.024s === RUN TestWarmPoolOnlyRoll I0503 19:25:19.297287 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.297378 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.297406 55041 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0503 19:25:19.297754 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.297776 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.298935 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.298967 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.304169 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.304217 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.304233 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.304244 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.305784 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.306343 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.306370 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.307503 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.307527 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.312736 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.312792 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.313569 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.313600 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.314700 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.314722 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.319949 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.319980 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.319993 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.320004 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.320661 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.321107 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.321129 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.322285 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.322305 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.327520 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.327553 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.328112 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.328134 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.329278 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.329299 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.334473 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.334521 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.335258 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.335283 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.336405 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.336434 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.341620 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.341653 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.341667 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.341677 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.342374 55041 instancegroups.go:407] Draining the node: "node-2a.local". I0503 19:25:19.342894 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.342918 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.344049 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.344078 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.349270 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.349328 55041 instancegroups.go:407] Draining the node: "node-2b.local". I0503 19:25:19.349921 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.349959 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.351103 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.351125 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.356313 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.356360 55041 instancegroups.go:407] Draining the node: "node-2c.local". I0503 19:25:19.356927 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.356950 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.358092 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.358112 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.363295 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.363333 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0503 19:25:19.363803 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.363839 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.365334 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.365382 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.365413 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.365425 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.365443 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.366553 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.366580 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.366588 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.366603 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.367698 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.367720 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.367750 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.367760 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.367775 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.368870 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.368901 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.368909 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.368953 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.370055 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.370085 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.370098 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.370115 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.371224 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.371239 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.371261 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.371270 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.371286 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.372383 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.372421 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.372434 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.372451 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.373565 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0503 19:25:19.373599 55041 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0503 19:25:19.373612 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.373635 55041 instancegroups.go:443] waiting for 1ms after terminating instance W0503 19:25:19.374767 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0503 19:25:19.374797 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0503 19:25:19.375050 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.375070 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.376184 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.376220 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.376242 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.376298 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.376322 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.376339 55041 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.376353 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.377188 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.377944 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.377973 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.379100 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.379133 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.379158 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.379196 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.379999 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.380029 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.381148 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.381179 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.381198 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.381217 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.381242 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.381264 55041 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.381280 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.382223 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.382988 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.383021 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.384154 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.384185 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.384205 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.384240 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.384973 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.385003 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.386126 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.386156 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.386174 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.386205 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.386900 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.386928 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.388000 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.388021 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.388033 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.388042 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.388053 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.388061 55041 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.388070 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.388707 55041 instancegroups.go:407] Draining the node: "node-2a.local". I0503 19:25:19.389225 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.389252 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.390375 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.390404 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.390425 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.390458 55041 instancegroups.go:407] Draining the node: "node-2b.local". I0503 19:25:19.391203 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.391231 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.392363 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.392397 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.392418 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.392451 55041 instancegroups.go:407] Draining the node: "node-2c.local". I0503 19:25:19.393243 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.393274 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.394389 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.394420 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0503 19:25:19.394442 55041 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0503 19:25:19.394464 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0503 19:25:19.394752 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0503 19:25:19.394984 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.395012 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.396130 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.396156 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.401322 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.401398 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.401413 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.401422 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.401947 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.402430 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.402455 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.403613 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.403641 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.408847 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.408915 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.409837 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.409873 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.410999 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.411027 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.416211 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.416252 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.416266 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.416276 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.416974 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.417491 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.417517 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.418645 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.418670 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.423841 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.423893 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.424443 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.424468 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.425598 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.425624 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.430853 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.430909 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.431578 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.431610 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.432752 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.432779 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.437984 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.438026 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.438069 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.438087 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.439068 55041 instancegroups.go:407] Draining the node: "node-2a.local". I0503 19:25:19.439571 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.439597 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.440748 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.440771 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.445984 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.446056 55041 instancegroups.go:407] Draining the node: "node-2b.local". I0503 19:25:19.446911 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.446945 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.448076 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.448111 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.453355 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.453432 55041 instancegroups.go:407] Draining the node: "node-2c.local". I0503 19:25:19.454365 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.454405 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.455558 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.455622 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.461133 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.461170 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0503 19:25:19.461315 55041 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0503 19:25:19.461615 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.461637 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.462759 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.462796 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.462814 55041 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0503 19:25:19.463125 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.463152 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.464287 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.464315 55041 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0503 19:25:19.464335 55041 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0503 19:25:19.464561 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.464588 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.464603 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.464620 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0503 19:25:19.464637 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0503 19:25:19.464806 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.464828 55041 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0503 19:25:19.464843 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.464862 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0503 19:25:19.464879 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0503 19:25:19.465089 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.465119 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.466252 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.466282 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.471502 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.471574 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.471611 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.471628 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.472513 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.473317 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.473361 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.474504 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.474550 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.474573 55041 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0503 19:25:19.474913 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.474944 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.476090 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.476121 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.481351 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.481429 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.481458 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.481475 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.482337 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.483114 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.483150 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.484292 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.484339 55041 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0503 19:25:19.484361 55041 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0503 19:25:19.484633 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.484650 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.484659 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.485378 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.485893 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.485924 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.487038 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.487064 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.492265 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.492338 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.492917 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.492942 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.494064 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.494099 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.499345 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.499405 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.500020 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.500045 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.501204 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.501240 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.506471 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.506528 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0503 19:25:19.506848 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.506874 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.506894 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.506916 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0503 19:25:19.506935 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0503 19:25:19.507105 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.507127 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.507142 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.507166 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0503 19:25:19.507184 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0503 19:25:19.507351 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.507373 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.507388 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.507404 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0503 19:25:19.507423 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0503 19:25:19.507565 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.507587 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.507600 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.508264 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.508735 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.508764 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.509904 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.509950 55041 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0503 19:25:19.509969 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.509985 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0503 19:25:19.509995 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0503 19:25:19.510182 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.510200 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.510210 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.510892 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.511423 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.511449 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.512576 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.512609 55041 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0503 19:25:19.512628 55041 instancegroups.go:482] Cluster did not validate within 0s E0503 19:25:19.512646 55041 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0503 19:25:19.512666 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0503 19:25:19.512904 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.512933 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.514082 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.514119 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.515226 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.520628 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.521789 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.527015 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.528164 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.533436 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.533526 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.533558 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.533575 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.534408 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.535220 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.535265 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.536403 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.536446 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.541650 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.541720 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.542526 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.542556 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.543708 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.543748 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.548953 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.549000 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.549022 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.549032 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.549696 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.550310 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.550333 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.551473 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.551505 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.556770 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.556836 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.557720 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.557752 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.558872 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.558910 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.564154 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.564215 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.565167 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.565199 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.566330 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.566360 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.571601 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.571644 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.571673 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.571689 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.572719 55041 instancegroups.go:407] Draining the node: "node-2a.local". I0503 19:25:19.573423 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.573455 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.574605 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.574637 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.579864 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.579923 55041 instancegroups.go:407] Draining the node: "node-2b.local". I0503 19:25:19.580816 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.580849 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.582014 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.582049 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.587273 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.587337 55041 instancegroups.go:407] Draining the node: "node-2c.local". I0503 19:25:19.588873 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.588905 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.590080 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.590115 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.595304 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.595336 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0503 19:25:19.595657 55041 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0503 19:25:19.595683 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.596849 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.596878 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.598015 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.599140 55041 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0503 19:25:19.600231 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.605453 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.605525 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.605543 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.605565 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.606115 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.606560 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.606585 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.607717 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.607755 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.613001 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.613063 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.614281 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.614319 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.615473 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.615517 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.620723 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.620778 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.620801 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.620818 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.621889 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.622650 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.622684 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.623793 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.623817 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.628969 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.629022 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.629641 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.629665 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.630788 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.630814 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.636404 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.636470 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.637408 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.637446 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.638577 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.638610 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.643804 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.643842 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.643865 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.643881 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.644934 55041 instancegroups.go:407] Draining the node: "node-2a.local". I0503 19:25:19.645717 55041 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0503 19:25:19.645753 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.646868 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.646891 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.652068 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.652118 55041 instancegroups.go:407] Draining the node: "node-2b.local". I0503 19:25:19.652739 55041 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0503 19:25:19.652768 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.653881 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.653900 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.659055 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.659093 55041 instancegroups.go:407] Draining the node: "node-2c.local". I0503 19:25:19.659633 55041 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0503 19:25:19.659657 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.660828 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.660848 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.666052 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.666100 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0503 19:25:19.666822 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.666848 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.666865 55041 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0503 19:25:19.667632 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.668441 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.668478 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.669597 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.669616 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.674822 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.674880 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.675783 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.675818 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.676942 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.676964 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.682279 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.682321 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0503 19:25:19.682620 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.682639 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.682648 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.683147 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:19.684119 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:19.684148 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.685181 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.685201 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.690772 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.690817 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:19.691404 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:19.691428 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.692552 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.692579 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.697799 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.697841 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0503 19:25:19.698211 55041 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W0503 19:25:19.698253 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0503 19:25:19.698276 55041 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W0503 19:25:19.698292 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0503 19:25:19.698312 55041 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W0503 19:25:19.698322 55041 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0503 19:25:19.698341 55041 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0503 19:25:19.698357 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0503 19:25:19.698580 55041 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I0503 19:25:19.698630 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:19.699791 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.699821 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.705035 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.705080 55041 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I0503 19:25:19.705139 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.705165 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.705181 55041 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0503 19:25:19.705906 55041 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I0503 19:25:19.705931 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.705954 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.705967 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:19.706744 55041 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0503 19:25:19.706803 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:19.707969 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.708006 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.714366 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.714407 55041 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I0503 19:25:19.714420 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.714435 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.714445 55041 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0503 19:25:19.715097 55041 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0503 19:25:19.715139 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:19.716297 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.716332 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:19.721572 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.721608 55041 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0503 19:25:19.721623 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0503 19:25:19.721851 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.721872 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.721882 55041 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0503 19:25:19.723168 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.723608 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.723636 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.724799 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.724843 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.724874 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:19.724925 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.725720 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.725874 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:19.725900 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.727055 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.727111 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.828019 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.828093 55041 instancegroups.go:407] Draining the node: "node-1e.local". I0503 19:25:19.828142 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:19.829327 55041 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0503 19:25:19.829588 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:19.829635 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.830798 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.830828 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.830850 55041 instancegroups.go:407] Draining the node: "node-1f.local". I0503 19:25:19.831327 55041 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0503 19:25:19.831355 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.832486 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.832553 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.932948 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.933020 55041 instancegroups.go:407] Draining the node: "node-1g.local". I0503 19:25:19.933823 55041 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0503 19:25:19.933862 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.935011 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.935044 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.935057 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0503 19:25:19.935247 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.935266 55041 instancegroups.go:510] Cluster validated. I0503 19:25:19.935275 55041 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0503 19:25:19.936321 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:19.936374 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:19.936778 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:19.937120 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:19.937144 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:19.938288 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:19.938325 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.038768 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.038829 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:20.038893 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:20.039933 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:20.040161 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:20.040201 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.041333 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.041373 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.041408 55041 instancegroups.go:407] Draining the node: "node-1e.local". I0503 19:25:20.041945 55041 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0503 19:25:20.041983 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.043125 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.043188 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.143573 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.143649 55041 instancegroups.go:407] Draining the node: "node-1f.local". I0503 19:25:20.144285 55041 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0503 19:25:20.144331 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.145464 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.145503 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.145523 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0503 19:25:20.145794 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.145822 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.145836 55041 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I0503 19:25:20.147851 55041 instancegroups.go:407] Draining the node: "master-1a.local". I0503 19:25:20.148513 55041 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0503 19:25:20.148553 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.149685 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.149725 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.149765 55041 instancegroups.go:407] Draining the node: "master-1c.local". I0503 19:25:20.149803 55041 instancegroups.go:407] Draining the node: "master-1b.local". I0503 19:25:20.150292 55041 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0503 19:25:20.150656 55041 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0503 19:25:20.150693 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.151809 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.151855 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.252288 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.252363 55041 instancegroups.go:407] Draining the node: "master-1e.local". I0503 19:25:20.252380 55041 instancegroups.go:407] Draining the node: "master-1d.local". I0503 19:25:20.253527 55041 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0503 19:25:20.253780 55041 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0503 19:25:20.253828 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.254969 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.255005 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.255038 55041 instancegroups.go:407] Draining the node: "master-1f.local". I0503 19:25:20.255869 55041 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0503 19:25:20.255914 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.257047 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.257098 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.357515 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.357585 55041 instancegroups.go:407] Draining the node: "master-1g.local". I0503 19:25:20.358216 55041 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0503 19:25:20.358262 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.359406 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.359434 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.359449 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0503 19:25:20.359642 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.359664 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.359672 55041 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0503 19:25:20.360733 55041 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0503 19:25:20.360774 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.361886 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.361918 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.361931 55041 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0503 19:25:20.361975 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.363082 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.363107 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.363134 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:20.363183 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.363630 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:20.364354 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.364392 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.365513 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.365551 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.466004 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.466067 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:20.466118 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:20.466704 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:20.467053 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:20.467081 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.468205 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.468242 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.468274 55041 instancegroups.go:407] Draining the node: "node-1e.local". I0503 19:25:20.469050 55041 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0503 19:25:20.469093 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.470233 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.470272 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.570667 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.570730 55041 instancegroups.go:407] Draining the node: "node-1f.local". I0503 19:25:20.571359 55041 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0503 19:25:20.571399 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.572522 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.572560 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.572580 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0503 19:25:20.572837 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.572867 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.572880 55041 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0503 19:25:20.574679 55041 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0503 19:25:20.574722 55041 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0503 19:25:20.574767 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.575908 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.575935 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.575960 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:20.576010 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.576480 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:20.576817 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.576842 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.577957 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.578002 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.678402 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.678479 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:20.678523 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:20.679466 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:20.679672 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:20.679711 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.680870 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.680901 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.680928 55041 instancegroups.go:407] Draining the node: "node-1e.local". I0503 19:25:20.681408 55041 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0503 19:25:20.681437 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.682546 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.682591 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.783324 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.783383 55041 instancegroups.go:407] Draining the node: "node-1f.local". I0503 19:25:20.784004 55041 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0503 19:25:20.784044 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.785191 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.785220 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.785233 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0503 19:25:20.785419 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.785434 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.785443 55041 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0503 19:25:20.785905 55041 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0503 19:25:20.785926 55041 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0503 19:25:20.785948 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.787081 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.787100 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:20.792294 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.792342 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:20.792379 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.793352 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.793387 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.793603 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:20.793633 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.794515 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.794542 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:20.799754 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.799776 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0503 19:25:20.799966 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.799982 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.799990 55041 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0503 19:25:20.800399 55041 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0503 19:25:20.800433 55041 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0503 19:25:20.800440 55041 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0503 19:25:20.800462 55041 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0503 19:25:20.800479 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.801598 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.801628 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:20.806871 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.806934 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:20.806986 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.807894 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.807917 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.808241 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:20.808283 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.809053 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.809087 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:20.814314 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.814349 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0503 19:25:20.814637 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.814676 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.814695 55041 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I0503 19:25:20.816312 55041 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0503 19:25:20.816389 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.817505 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.817554 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.817570 55041 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0503 19:25:20.817635 55041 instancegroups.go:179] waiting for 1ms after detaching instance I0503 19:25:20.818828 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.818855 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.818885 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:20.818936 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.818978 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:20.820163 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:20.820207 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.820385 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.820422 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.820576 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:20.820601 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.821336 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.922072 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.922132 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:20.922764 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:20.922799 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.923947 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.923976 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.923989 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0503 19:25:20.924168 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.924187 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.924195 55041 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0503 19:25:20.925402 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:20.925872 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:20.925898 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.927036 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.927061 55041 instancegroups.go:510] Cluster validated. I0503 19:25:20.927085 55041 instancegroups.go:407] Draining the node: "node-1e.local". I0503 19:25:20.927118 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:20.927938 55041 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0503 19:25:20.928091 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:20.928114 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:20.929250 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:20.929289 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.029938 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.030000 55041 instancegroups.go:407] Draining the node: "node-1g.local". I0503 19:25:21.030046 55041 instancegroups.go:407] Draining the node: "node-1f.local". I0503 19:25:21.030665 55041 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0503 19:25:21.031005 55041 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0503 19:25:21.031033 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.032159 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.032198 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.032230 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:21.032994 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:21.033030 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.034175 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.034213 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.134841 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.134902 55041 instancegroups.go:407] Draining the node: "node-1d.local". I0503 19:25:21.135517 55041 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0503 19:25:21.135553 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.136687 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.136725 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.136745 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0503 19:25:21.137004 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.137031 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.137046 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0503 19:25:21.137065 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0503 19:25:21.137081 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0503 19:25:21.137104 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0503 19:25:21.137250 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.137272 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.137285 55041 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0503 19:25:21.138210 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0503 19:25:21.138235 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0503 19:25:21.138251 55041 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0503 19:25:21.138290 55041 instancegroups.go:407] Draining the node: "node-1a.local". I0503 19:25:21.139063 55041 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0503 19:25:21.139094 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.140240 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.140259 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:21.145460 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.145519 55041 instancegroups.go:407] Draining the node: "node-1b.local". I0503 19:25:21.146427 55041 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0503 19:25:21.146463 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.147628 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.147660 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:21.152979 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.153014 55041 instancegroups.go:407] Draining the node: "node-1c.local". I0503 19:25:21.153527 55041 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0503 19:25:21.153548 55041 instancegroups.go:443] waiting for 1ms after terminating instance I0503 19:25:21.154690 55041 instancegroups.go:477] Validating the cluster. I0503 19:25:21.154712 55041 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0503 19:25:21.159905 55041 instancegroups.go:510] Cluster validated. I0503 19:25:21.159950 55041 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.961s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.004s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.012s === RUN TestParse E0503 19:25:19.214084 55044 versions.go:96] unable to parse Kubernetes version "" E0503 19:25:19.214204 55044 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.038s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.14s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.219s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.103s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.082s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.087s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0503 19:25:19.274656 55054 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.078s === RUN Test_RunEtcdManagerBuilder I0503 19:25:19.290129 55056 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0503 19:25:19.337619 55056 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W0503 19:25:19.338853 55056 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.141s === RUN Test_RunKubeApiserverBuilder I0503 19:25:19.278129 55058 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.100s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.106s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.035s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.095s === RUN TestServerGroupBuilder I0503 19:25:27.009040 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.020861 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.034778 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.049666 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.102036 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.128545 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.150684 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.163621 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.177607 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.296439 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.304339 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.313192 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.320425 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.328036 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.334835 55857 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.33s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.395s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.023s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.011s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.023s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.00s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.64s) --- PASS: TestIssueCert/ca (0.43s) --- PASS: TestIssueCert/client (0.59s) --- PASS: TestIssueCert/clientOneYear (0.62s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.658s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.003s === RUN TestAddUntaggedRouteTables I0503 19:25:27.158336 56060 routetable.go:53] DescribeRouteTables: { } W0503 19:25:27.158495 56060 tags.go:67] cluster tag not found on route-table:rtb-1234 I0503 19:25:27.158518 56060 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0503 19:25:27.158696 56060 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0503 19:25:27.158802 56060 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0503 19:25:27.158931 56060 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0503 19:25:27.159070 56060 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0503 19:25:27.159178 56060 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0503 19:25:27.159290 56060 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0503 19:25:27.159431 56060 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0503 19:25:27.159493 56060 volumes.go:128] DescribeVolumes: { } W0503 19:25:27.159540 56060 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0503 19:25:27.159757 56060 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:27.159885 56060 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:27.159963 56060 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:27.160078 56060 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:27.160159 56060 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:27.160219 56060 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:27.160252 56060 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:27.160317 56060 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:27.160359 56060 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:27.160398 56060 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0503 19:25:27.160443 56060 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0503 19:25:27.160484 56060 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0503 19:25:27.160538 56060 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0503 19:25:27.160582 56060 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0503 19:25:27.160637 56060 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0503 19:25:27.160694 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0503 19:25:27.160741 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0503 19:25:27.160783 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0503 19:25:27.160821 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0503 19:25:27.160864 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0503 19:25:27.160903 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0503 19:25:27.160947 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0503 19:25:27.160981 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0503 19:25:27.161023 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0503 19:25:27.161075 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0503 19:25:27.161125 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0503 19:25:27.161163 56060 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.084s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.034s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.076s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.005s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.005s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.006s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.003s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.040s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.079s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.006s === RUN TestRemovesDuplicateGuardedBlocks W0503 19:25:35.247982 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248045 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248054 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248062 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248070 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248078 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248085 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248093 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248101 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248108 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248115 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248123 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248130 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248137 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.248144 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.248151 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.268610 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.268651 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.268775 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.268794 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.268889 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.268908 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269010 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269030 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269122 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269141 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269237 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269257 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269371 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269390 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269509 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269528 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269656 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269676 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269771 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269789 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.269896 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.269915 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270002 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270020 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270108 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270128 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270221 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270240 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270330 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270349 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270442 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270461 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270549 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270567 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270649 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270666 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270749 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270766 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.270905 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.270924 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271012 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271030 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271118 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271135 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271218 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271237 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271328 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271347 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271436 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271462 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271546 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271564 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271646 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271663 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271743 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271761 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271844 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271862 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.271965 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.271985 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272073 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272092 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272189 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272209 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272322 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272341 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272421 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272438 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272516 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272533 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272641 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272660 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272750 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272769 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272881 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.272900 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.272988 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273006 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273103 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273121 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273234 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273252 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273335 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273352 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273435 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273453 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273536 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273555 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273640 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273659 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273750 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273769 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273858 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.273876 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.273979 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274004 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274086 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274103 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274181 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274199 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274288 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274306 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274391 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274409 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274507 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274526 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274618 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274636 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274720 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274737 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274819 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274836 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.274915 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.274931 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275014 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275031 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275118 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275136 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275214 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275230 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275311 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275329 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275419 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275436 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275546 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275564 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275667 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275686 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275769 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275787 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.275875 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.275893 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276001 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276020 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276106 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276125 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276216 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276234 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276315 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276332 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276412 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276429 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276512 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276529 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276610 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276628 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276715 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276732 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276814 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276831 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.276918 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.276937 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277026 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277044 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277127 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277145 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277237 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277254 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277346 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277365 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277456 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277473 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277556 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277573 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277652 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277668 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277797 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277816 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.277912 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.277931 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.278016 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.278035 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.278120 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.278772 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.278871 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.278888 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.278982 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279000 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279083 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279101 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279183 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279201 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279283 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279300 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279384 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279402 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279489 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279507 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279596 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279615 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279704 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279721 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279801 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279818 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279897 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.279914 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.279991 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280008 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0503 19:25:35.280500 57315 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0503 19:25:35.280515 57315 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0503 19:25:35.280529 57315 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0503 19:25:35.280542 57315 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0503 19:25:35.280555 57315 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0503 19:25:35.280567 57315 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0503 19:25:35.280577 57315 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0503 19:25:35.280611 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280627 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280640 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.280653 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280666 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.280680 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280694 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.280707 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280721 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.280733 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280746 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280760 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280775 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280788 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.280801 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.280815 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298230 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298260 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298324 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298344 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298398 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298409 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298462 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298472 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298569 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298580 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298637 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298648 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298701 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298711 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298762 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298772 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298821 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298832 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298884 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298894 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.298967 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.298979 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299037 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299048 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299108 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299120 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299185 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299196 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299246 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299256 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299306 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299325 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299378 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299389 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299438 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299448 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299501 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299512 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299566 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299577 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299635 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299646 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299698 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299708 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299760 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299770 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299819 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299829 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299878 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299888 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.299951 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.299963 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300012 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300022 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300075 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300086 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300157 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300169 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300221 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300231 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300281 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300292 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300341 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300351 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300401 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300411 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300465 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300475 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300525 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300535 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300586 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300596 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300645 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300655 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300705 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300715 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300768 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300780 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300829 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300839 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300889 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300899 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.300956 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.300967 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301017 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301028 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301079 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301090 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301139 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301149 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301198 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301208 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301260 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301271 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301320 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301330 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301381 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301392 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301440 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301450 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301499 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301509 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301562 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301572 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301638 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301649 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301701 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301712 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301761 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301771 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301820 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301830 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301878 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301888 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.301941 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.301951 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302000 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302011 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302067 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302078 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302128 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302139 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302192 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302202 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302270 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302283 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302338 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302349 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302401 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302412 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302461 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302471 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302521 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302532 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302582 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302592 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302642 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302652 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302702 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302712 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302779 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302790 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302840 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302850 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302899 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302909 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.302959 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.302968 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303019 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303030 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303082 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303092 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303163 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303175 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303225 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303235 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303285 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303297 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303346 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303356 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303407 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303417 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303467 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303477 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303528 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303538 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303591 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303601 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303651 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303663 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303713 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303723 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303772 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303782 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303832 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303843 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303892 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303904 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.303955 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.303965 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304014 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304025 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304077 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304087 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304136 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304146 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304197 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304207 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304256 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304266 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304314 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304324 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304374 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304384 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304432 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304443 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0503 19:25:35.304492 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0503 19:25:35.304502 57315 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.061s ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.079s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.035s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.057s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.03s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.00s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.111s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0503 19:25:37.946722 57863 files.go:57] Writing file "/tmp/TestWriteFile2463747056/001/SubDir/test1.tmp" I0503 19:25:37.946877 57863 files.go:113] Changing file mode for "/tmp/TestWriteFile2463747056/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.01s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.049s === RUN TestBootstrapChannelBuilder_BuildTasks I0503 19:25:37.992097 57864 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:37.992292 57864 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:37.992355 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:37.992403 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:37.992455 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:37.992496 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:37.992521 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:37.992554 57864 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:37.992588 57864 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:38.006057 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.080756 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.236296 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.342708 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.421329 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.501734 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.563216 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.628404 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:38.713849 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:39.172192 57864 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.23s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0503 19:25:39.220146 57864 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:39.220227 57864 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:39.220284 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:39.220319 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:39.220354 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:39.220384 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:39.220401 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:39.220425 57864 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:39.220442 57864 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:39.220461 57864 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0503 19:25:39.221665 57864 zones.go:93] ListHostedZonesPages { } I0503 19:25:39.287984 57864 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.07s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0503 19:25:39.288189 57864 featureflag.go:160] FeatureFlag "ImageDigest"=false I0503 19:25:39.288301 57864 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:39.288378 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0503 19:25:39.288531 57864 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0503 19:25:39.288634 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0503 19:25:39.288723 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0503 19:25:39.288791 57864 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0503 19:25:39.288868 57864 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0503 19:25:39.288945 57864 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0503 19:25:39.290951 57864 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.11s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.18s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0503 19:25:39.601035 57864 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0503 19:25:39.633871 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.633898 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.633909 57864 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0503 19:25:39.633928 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0503 19:25:39.633938 57864 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0503 19:25:39.633956 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0503 19:25:39.633974 57864 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0503 19:25:39.633992 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.634003 57864 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0503 19:25:39.634020 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.634031 57864 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0503 19:25:39.634048 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.634071 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.634089 57864 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0503 19:25:39.634099 57864 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0503 19:25:39.634108 57864 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0503 19:25:39.634117 57864 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0503 19:25:39.634128 57864 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0503 19:25:39.634138 57864 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0503 19:25:39.634148 57864 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0503 19:25:39.634160 57864 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0503 19:25:39.635194 57864 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.747s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0503 19:25:37.971561 57865 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3493144692/001 I0503 19:25:37.972912 57865 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3493144692/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0503 19:25:37.982593 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0503 19:25:37.982712 57865 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0503 19:25:37.982833 57865 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0503 19:25:37.983041 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.983311 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.983416 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.983449 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.983745 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:37.983807 57865 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0503 19:25:37.983889 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.984010 57865 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0503 19:25:37.984152 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0503 19:25:37.984286 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.984393 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.984463 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.984493 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.984560 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.984638 57865 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0503 19:25:37.984750 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0503 19:25:37.985088 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:37.985221 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.985402 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:37.985472 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:37.985511 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.985580 57865 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0503 19:25:37.985829 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:37.986116 57865 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0503 19:25:37.986197 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:37.986239 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:37.986298 57865 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0503 19:25:37.986400 57865 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:37.986647 57865 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0503 19:25:37.986732 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:37.986774 57865 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" },{ Key: "AssociatedElasticIp", Value: "192.0.2.1" }] } I0503 19:25:37.986856 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0503 19:25:37.987000 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:37.987076 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.987152 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.987181 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.987229 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.987269 57865 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0503 19:25:37.987330 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:37.987425 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:37.987466 57865 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0503 19:25:37.987525 57865 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:37.987618 57865 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0503 19:25:37.987662 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0503 19:25:37.987737 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0503 19:25:37.987866 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0503 19:25:37.987928 57865 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0503 19:25:37.987980 57865 internetgateways.go:66] CreateInternetGateway: { } I0503 19:25:37.988007 57865 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0503 19:25:37.988125 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.988189 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.988248 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.988275 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.988393 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:37.988432 57865 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0503 19:25:37.988479 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.988568 57865 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0503 19:25:37.988676 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0503 19:25:37.988780 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.988835 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.988892 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.988926 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.988980 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.989041 57865 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0503 19:25:37.989136 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0503 19:25:37.992228 57865 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3470699951/001 I0503 19:25:37.994966 57865 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3470699951/001 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0503 19:25:37.995887 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.995986 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.996212 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:37.996303 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:37.996352 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.996430 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.996570 57865 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0503 19:25:37.996902 57865 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0503 19:25:37.997018 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0503 19:25:37.997128 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0503 19:25:37.997331 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.997437 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.997526 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.997559 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.997617 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.997691 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.997776 57865 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0503 19:25:37.997913 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0503 19:25:37.998469 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.998561 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.998715 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:37.998781 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:37.998821 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.998862 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.998955 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:37.999138 57865 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0503 19:25:37.999210 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:37.999253 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:37.999299 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0503 19:25:37.999429 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:37.999496 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:37.999570 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:37.999603 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:37.999648 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:37.999690 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:37.999776 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:37.999872 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:37.999912 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0503 19:25:38.000158 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:38.000227 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.000412 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:38.000502 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:38.000566 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.000641 57865 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0503 19:25:38.000932 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.001160 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:38.001439 57865 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:38.001425 57865 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0503 19:25:38.001577 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:38.001644 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.001708 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0503 19:25:38.001860 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:38.001954 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.002028 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.002057 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.002102 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.002143 57865 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0503 19:25:38.002208 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:38.002325 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.002361 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.002443 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0503 19:25:38.002673 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:38.002744 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.002894 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:38.002963 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:38.003004 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.003046 57865 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0503 19:25:38.003112 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:38.003270 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.003378 57865 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0503 19:25:38.003391 57865 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0503 19:25:38.003472 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:38.003515 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.003554 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0503 19:25:38.003690 57865 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0503 19:25:38.003757 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.003831 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.003865 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.003914 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.003957 57865 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0503 19:25:38.004017 57865 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0503 19:25:38.004129 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.004157 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.004203 57865 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0503 19:25:38.004297 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0503 19:25:38.004360 57865 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0503 19:25:38.004454 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:38.004497 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.004536 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.004551 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.004640 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:38.004676 57865 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0503 19:25:38.004710 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:38.004754 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.004867 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0503 19:25:38.004894 57865 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0503 19:25:38.004928 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0503 19:25:38.005023 57865 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0503 19:25:38.005062 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.005111 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.005126 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.005156 57865 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0503 19:25:38.005198 57865 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0503 19:25:38.005258 57865 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0503 19:25:38.005380 57865 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:38.005427 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.005528 57865 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0503 19:25:38.005566 57865 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0503 19:25:38.005610 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.005635 57865 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:38.005694 57865 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:38.005739 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0503 19:25:38.005782 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.005797 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.005825 57865 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0503 19:25:38.005850 57865 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0503 19:25:38.005996 57865 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0503 19:25:38.006075 57865 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:38.006114 57865 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0503 19:25:38.006163 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0503 19:25:38.006178 57865 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0503 19:25:38.006208 57865 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.101s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.056s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.032s === RUN TestDiskRenderAzure I0503 19:25:37.943774 57872 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0503 19:25:37.944434 57872 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0503 19:25:37.944864 57872 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0503 19:25:37.945656 57872 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0503 19:25:37.946106 57872 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0503 19:25:37.946531 57872 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0503 19:25:37.946874 57872 resourcegroup.go:103] Creating a new Resource Group with name: rg I0503 19:25:37.946892 57872 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0503 19:25:37.947159 57872 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0503 19:25:37.947906 57872 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0503 19:25:37.948248 57872 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0503 19:25:37.949180 57872 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0503 19:25:37.949495 57872 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0503 19:25:37.949744 57872 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.042s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.039s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestProjectIAMBinding I0503 19:25:40.764827 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.765360 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.765410 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.765844 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.765888 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.766034 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0503 19:25:40.766153 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.766227 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.766264 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.766385 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.766443 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.766494 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0503 19:25:40.766593 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.766821 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.766864 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.767112 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0503 19:25:40.767160 58996 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0503 19:25:40.767277 58996 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.054s === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.042s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.033s === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.034s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.006s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.025s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0503 19:25:44.806123 59269 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0503 19:25:44.806187 59269 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0503 19:25:44.806204 59269 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0503 19:25:44.806219 59269 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0503 19:25:44.806230 59269 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0503 19:25:44.806241 59269 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0503 19:25:44.806253 59269 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0503 19:25:44.806693 59269 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.059s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.006s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.007s === RUN TestGetProxyEnvVars W0503 19:25:44.772413 59270 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.022s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.006s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.006s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.005s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.025s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0503 19:25:46.915681 59361 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.058s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 134.4 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r5.apk >>> kops: Build complete at Wed, 03 May 2023 19:25:50 +0000 elapsed time 0h 3m 20s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20230503.192232) (2/3) Purging kubernetes (1.26.1-r4) (3/3) Purging go (1.20.4-r0) Executing busybox-1.36.0-r8.trigger OK: 393 MiB in 101 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...