>>> kops: Building testing/kops 1.25.1-r1 (using abuild 3.10.0_rc1-r2) started Wed, 02 Nov 2022 00:55:58 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r3) (2/3) Installing go (1.19.3-r0) (3/3) Installing .makedepends-kops (20221102.005559) Executing busybox-1.35.0-r27.trigger OK: 801 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 30.8M 100 30.8M 0 0 35.9M 0 --:--:-- --:--:-- --:--:-- 35.9M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.1.tar.gz >>> kops: Checking sha512sums... kops-1.25.1.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.1.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.103 go: downloading k8s.io/apimachinery v0.25.2 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.9.1 go: downloading google.golang.org/api v0.97.0 go: downloading github.com/spf13/cobra v1.5.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading github.com/spf13/viper v1.13.0 go: downloading k8s.io/api v0.25.2 go: downloading k8s.io/cli-runtime v0.25.2 go: downloading k8s.io/klog v1.0.0 go: downloading k8s.io/client-go v0.25.2 go: downloading golang.org/x/crypto v0.0.0-20220919173607-35f4265a4bc0 go: downloading github.com/stretchr/testify v1.8.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/utils v0.0.0-20220922104903-7796b5f52b7e go: downloading cloud.google.com/go/compute v1.10.0 go: downloading golang.org/x/oauth2 v0.0.0-20220909003341-f21342109be1 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.2 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading helm.sh/helm/v3 v3.10.0 go: downloading k8s.io/kubectl v0.25.2 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.8.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading sigs.k8s.io/controller-runtime v0.13.0 go: downloading github.com/prometheus/client_golang v1.13.0 go: downloading k8s.io/component-base v0.25.2 go: downloading golang.org/x/net v0.0.0-20220921203646-d300de134e69 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/google/go-cmp v0.5.9 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/digitalocean/godo v1.85.0 go: downloading golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8 go: downloading github.com/hetznercloud/hcloud-go v1.35.3 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.2 go: downloading k8s.io/mount-utils v0.25.2 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading k8s.io/cloud-provider-aws v1.25.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/spotinst/spotinst-sdk-go v1.129.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.0.0-20220907140024-f12130a52804 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.14.0 go: downloading github.com/zclconf/go-cty v1.11.0 go: downloading k8s.io/gengo v0.0.0-20220913193501-391367153a38 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading golang.org/x/text v0.3.7 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/pkg/errors v0.9.1 go: downloading google.golang.org/appengine v1.6.7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.2 go: downloading golang.org/x/time v0.0.0-20220609170525-579cf78fd858 go: downloading k8s.io/apiextensions-apiserver v0.25.0 go: downloading sigs.k8s.io/gateway-api v0.4.3 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.20 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading google.golang.org/grpc v1.48.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading k8s.io/csi-translation-lib v0.25.2 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/google/btree v1.0.1 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading google.golang.org/genproto v0.0.0-20220915135415-7fd63a7952de go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.1 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.1.6 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.0 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/locker v1.0.1 go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.1/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.1/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I1102 00:58:05.234109 30120 addon.go:233] addon test wants to update all nodes I1102 00:58:05.235463 30120 addon.go:233] addon test wants to update worker nodes I1102 00:58:05.235746 30120 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I1102 00:58:05.236120 30120 addon.go:266] installing PKI for "test" I1102 00:58:07.161649 30120 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (3.26s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 3.358s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.115s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I1102 00:58:20.241942 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.242202 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.242259 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.242294 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.242337 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.242435 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.242457 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.242484 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.242506 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.242574 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.242603 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.281184 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.281341 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.281636 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.293274 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.293435 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.293467 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.293500 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.293555 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.293600 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.293613 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.293633 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.293649 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.294187 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.294250 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.347808 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.347983 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.348267 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.353258 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.353417 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.353455 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.353481 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.353510 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.353559 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.353577 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.353606 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.353619 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.353678 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.353700 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.373704 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.373854 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.374124 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.379253 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.379385 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.379419 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.379460 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.379490 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.379516 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.379530 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.379551 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.379566 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.379624 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.379648 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.419957 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.420098 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.420333 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.427346 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.427544 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.427590 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.427624 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.427663 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.427696 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.427718 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.427757 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.427778 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.427853 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.427890 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.460500 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.460766 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.461167 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.469524 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.469716 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.469758 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.469794 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.469836 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.469869 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.469893 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.469925 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.469947 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.470021 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.470056 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.502910 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.503077 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.503336 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.507985 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.508154 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.508212 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.508238 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.508275 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.508294 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.508319 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.508337 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.508350 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.508444 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.508466 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.552059 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.552202 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.552443 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.557379 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.557517 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.557548 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.557584 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.557611 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.557635 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.557650 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.557670 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.557684 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.557735 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.557756 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.594121 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.594224 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.594593 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.36s) === RUN TestCreateClusterHetzner I1102 00:58:20.600640 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.600797 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.600828 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.600857 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.600885 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.600932 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.600948 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.600970 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.600985 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.601041 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.601063 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1102 00:58:20.627036 34543 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1102 00:58:20.627109 34543 new_cluster.go:1279] Cloud Provider ID = hetzner I1102 00:58:20.635793 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.635932 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.635964 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.635989 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.636016 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.636042 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.636058 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.636078 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.636092 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.636151 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.636182 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.735132 34543 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.14s) === RUN TestCreateClusterOverride I1102 00:58:20.739746 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.739899 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.739931 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.739956 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.739983 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.740028 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.740042 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.740061 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.740076 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.740126 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.740148 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.767111 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.767512 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.767732 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.03s) === RUN TestCreateClusterComplex I1102 00:58:20.772184 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.772342 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.772369 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.772391 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.772416 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.772459 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.772472 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.772488 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.772503 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.772552 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.772573 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.790261 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.790412 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.790651 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.02s) === RUN TestCreateClusterHA I1102 00:58:20.794907 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.795063 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.795091 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.795114 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.795137 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.795183 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.795196 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.795215 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.795229 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.795279 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.795300 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.832122 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.832416 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.832430 34543 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1102 00:58:20.832437 34543 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1102 00:58:20.832691 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:20.841466 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.841624 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.841658 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.841685 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.841713 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.841735 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.841749 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.841768 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.841782 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.841844 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.841867 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.861941 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.862207 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.862220 34543 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1102 00:58:20.862229 34543 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1102 00:58:20.862497 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.07s) === RUN TestCreateClusterHAGCE I1102 00:58:20.869359 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.869514 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.869545 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.869571 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.869599 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.869645 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.869661 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.869680 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.869695 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.869749 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.869770 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.892593 34543 new_cluster.go:1279] Cloud Provider ID = gce I1102 00:58:20.892730 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.03s) === RUN TestCreateClusterGCE I1102 00:58:20.899784 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.899938 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.899968 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.899990 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.900019 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.900064 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.900079 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.900098 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.900112 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.900162 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.900183 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.926827 34543 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1102 00:58:20.926944 34543 new_cluster.go:1279] Cloud Provider ID = gce I1102 00:58:20.927066 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.03s) === RUN TestCreateClusterHASharedZone I1102 00:58:20.932796 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.932952 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.932985 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.933015 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.933043 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.933090 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.933105 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.933123 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.933138 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.933192 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.933214 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1102 00:58:20.969221 34543 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1102 00:58:20.969284 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:20.969484 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:20.969746 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.04s) === RUN TestCreateClusterHASharedZones I1102 00:58:20.975333 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.975509 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.975535 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.975569 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.975593 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.975648 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.975662 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.975690 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.975705 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.975765 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:20.975786 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1102 00:58:21.049443 34543 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1102 00:58:21.049498 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.049845 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.049860 34543 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1102 00:58:21.050190 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.08s) === RUN TestCreateClusterPrivate I1102 00:58:21.060372 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.060540 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.060572 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.060608 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.060633 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.060690 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.060704 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.060734 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.060750 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.060799 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.060821 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.132014 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.132318 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.132340 34543 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1102 00:58:21.132743 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.132783 34543 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.08s) === RUN TestCreateClusterPrivateGCE I1102 00:58:21.141442 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.141692 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.141760 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.141822 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.141884 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.141955 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.141988 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.142024 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.142047 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.142117 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.142145 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.174238 34543 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1102 00:58:21.174315 34543 new_cluster.go:1279] Cloud Provider ID = gce I1102 00:58:21.174400 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.04s) === RUN TestCreateClusterWithNGWSpecified I1102 00:58:21.182135 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.182289 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.182330 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.182354 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.182380 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.182436 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.182450 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.182480 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.182494 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.182553 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.182574 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.206740 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.206934 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.206957 34543 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1102 00:58:21.207209 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.207236 34543 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.03s) === RUN TestCreateClusterWithINGWSpecified I1102 00:58:21.212281 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.212423 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.212449 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.212470 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.212494 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.212538 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.212551 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.212568 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.212582 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.212634 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.212654 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.247950 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.248123 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.248137 34543 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1102 00:58:21.248392 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.248419 34543 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.04s) === RUN TestCreateClusterSharedVPC I1102 00:58:21.253557 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.253751 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.253793 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.253814 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.253852 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.253899 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.253926 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.253947 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.253960 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.254031 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.254052 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.301356 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.301504 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.301566 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.301614 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.301626 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.301666 34543 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I1102 00:58:21.301928 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.05s) === RUN TestCreateClusterSharedSubnets I1102 00:58:21.308457 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.308618 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.308659 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.308682 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.308709 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.308766 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.308779 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.308796 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.308821 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.308882 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.308903 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.358592 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.358661 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.358760 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.358966 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.359003 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.359034 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.359054 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.359445 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.06s) === RUN TestCreateClusterSharedSubnetsVpcLookup I1102 00:58:21.365105 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.365264 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.365298 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.365323 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.365354 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.365400 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.365418 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.365438 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.365456 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.365513 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.365537 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.411238 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.411281 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.411299 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.411365 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.411483 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.411509 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.411528 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.411539 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.411816 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.05s) === RUN TestCreateClusterPrivateSharedSubnets I1102 00:58:21.416229 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.416381 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.416409 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.416437 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.416465 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.416511 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.416524 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.416543 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.416557 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.416626 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.416653 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.438480 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.438543 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.438637 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.438664 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.438710 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.438885 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.438931 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.438958 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:58:21.438977 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:21.439422 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.03s) === RUN TestCreateClusterIPv6 I1102 00:58:21.446317 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.446538 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.446588 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.446625 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.446666 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.446737 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.446762 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.446792 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.446816 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.446907 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.446952 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.469215 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.469382 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.469714 34543 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.03s) === RUN TestCreateClusterDifferentAMIs I1102 00:58:21.474902 34543 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1102 00:58:21.474919 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.475065 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.475094 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.475117 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.475143 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.475193 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.475209 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.475230 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.475246 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.475297 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.475320 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.523957 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.524035 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.524048 34543 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1102 00:58:21.524298 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.530270 34543 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.06s) === RUN TestCreateClusterKarpenter I1102 00:58:21.530376 34543 featureflag.go:160] FeatureFlag "Karpenter"=true I1102 00:58:21.530387 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.530532 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.530562 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.530587 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.530613 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.530658 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.530671 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.530688 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.530703 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.530755 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.530807 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1102 00:58:21.580278 34543 new_cluster.go:1279] Cloud Provider ID = aws I1102 00:58:21.580456 34543 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1102 00:58:21.580473 34543 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1102 00:58:21.580485 34543 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1102 00:58:21.580770 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.585682 34543 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.06s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I1102 00:58:21.586002 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.586147 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.586182 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.586239 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.586277 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.586305 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.586326 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.586356 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.586375 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.590445 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I1102 00:58:21.591362 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.591503 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.591531 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.591561 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.591595 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.591636 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.591659 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.591687 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.591705 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.604439 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:21.606108 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:21.606179 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:24.712009 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:28.191920 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:28.192936 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:28.194086 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:28.194125 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:28.194301 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:28.194630 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:28.194860 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:28.195253 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:28.200000 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:28.201207 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.201409 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.205632 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.206289 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.206415 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:28.206983 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:28.207246 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:28.245977 34543 target.go:112] Terraform output is in /tmp/test3875655761/out I1102 00:58:28.248164 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:28.248311 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:28.248354 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:28.248389 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:28.248422 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:28.248454 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:28.248471 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:28.248495 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:28.248514 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:28.271740 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:28.274745 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.274842 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:28.314330 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:28.388931 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:58:28.389210 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:28.389426 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:28.389481 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:28.389712 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:28.389742 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:28.389849 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:28.389893 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:28.390011 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:28.389892 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:58:28.392205 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:28.395421 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:28.395802 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.395980 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.396634 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.398036 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.398173 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:28.398470 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:28.398666 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:28.403084 34543 target.go:168] Cloudformation output is in /tmp/test2832820543/out --- PASS: TestMinimal (6.82s) === RUN TestMinimal_v1_23 I1102 00:58:28.412232 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:28.412407 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:28.412445 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:28.412508 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:28.412542 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:28.412573 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:28.412591 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:28.412614 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:28.412631 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:28.428355 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1102 00:58:28.430691 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:28.430758 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.583052 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:29.701301 34543 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1102 00:58:29.701591 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:29.701632 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:29.701741 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:29.701822 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:29.701991 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:29.702274 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:29.704321 34543 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1102 00:58:29.710160 34543 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1102 00:58:29.710413 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.710686 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.713847 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.715756 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.715896 34543 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1102 00:58:29.716165 34543 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1102 00:58:29.716334 34543 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1102 00:58:29.758957 34543 target.go:112] Terraform output is in /tmp/test2364904060/out I1102 00:58:29.761335 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:29.761475 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:29.761516 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:29.761550 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:29.761585 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:29.761615 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:29.761634 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:29.761657 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:29.761676 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:29.785939 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:29.788952 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.789048 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:29.827352 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:29.919479 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:29.919972 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1102 00:58:29.920021 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:29.920150 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:29.920047 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:29.920188 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:29.920343 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:29.920548 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:29.920578 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:29.920708 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1102 00:58:29.922626 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:29.925579 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:29.925718 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.926021 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.926355 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.928316 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.928808 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:29.929033 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:29.929170 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:29.931984 34543 target.go:168] Cloudformation output is in /tmp/test4273296749/out --- PASS: TestMinimal_v1_23 (1.53s) === RUN TestMinimal_v1_24 I1102 00:58:29.943957 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:29.944202 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:29.944258 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:29.944351 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:29.944401 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:29.944446 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:29.944475 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:29.944511 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:29.944540 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:29.970826 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:29.974099 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:29.974192 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.234962 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:31.382206 34543 executor.go:111] Tasks: 0 done / 93 total; 46 can run W1102 00:58:31.382538 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:31.382634 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:31.382770 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:31.382884 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:31.382831 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:31.382984 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:31.384791 34543 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1102 00:58:31.393220 34543 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1102 00:58:31.393758 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.393964 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.394976 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.395165 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.395280 34543 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1102 00:58:31.395542 34543 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1102 00:58:31.395681 34543 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1102 00:58:31.438999 34543 target.go:112] Terraform output is in /tmp/test4199863898/out I1102 00:58:31.443161 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:31.443382 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:31.443442 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:31.443495 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:31.443533 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:31.443577 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:31.443598 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:31.443623 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:31.443641 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:31.465221 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:31.468366 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.468475 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:31.506576 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:31.605057 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:58:31.605733 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:31.605893 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:31.606041 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:31.606069 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:31.606056 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:31.606122 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:31.606163 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:58:31.606254 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:31.606263 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:31.608337 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:31.613593 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:31.613897 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.614083 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.615114 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.616383 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.616524 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:31.616810 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:31.617017 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:31.619962 34543 target.go:168] Cloudformation output is in /tmp/test795983143/out --- PASS: TestMinimal_v1_24 (1.69s) === RUN TestMinimal_v1_25 I1102 00:58:31.634122 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:31.634487 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:31.634581 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:31.634735 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:31.634792 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:31.634838 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:31.634867 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:31.634902 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:31.634930 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:31.660638 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:31.663968 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:31.664062 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:32.939172 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:33.120744 34543 executor.go:111] Tasks: 0 done / 93 total; 46 can run W1102 00:58:33.121264 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:33.122490 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:33.122532 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:33.122637 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:33.122802 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:33.122759 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:33.128839 34543 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1102 00:58:33.134764 34543 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1102 00:58:33.135165 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.135352 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.136539 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.137935 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.138058 34543 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1102 00:58:33.138339 34543 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1102 00:58:33.138594 34543 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1102 00:58:33.192188 34543 target.go:112] Terraform output is in /tmp/test3305948458/out I1102 00:58:33.197072 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:33.197309 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:33.197366 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:33.197415 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:33.197472 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:33.197517 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:33.197548 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:33.197586 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:33.197616 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:33.220967 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:33.223983 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.224084 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:33.262116 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:33.361547 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:33.361859 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:33.362067 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:33.362107 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:33.362337 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:33.363356 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:33.363401 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:33.363497 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:33.363517 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:33.362261 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:58:33.368781 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:33.374293 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:33.374837 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.375072 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.376327 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.377234 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.377372 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:33.377632 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:33.377821 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:33.380772 34543 target.go:168] Cloudformation output is in /tmp/test2574046511/out --- PASS: TestMinimal_v1_25 (1.76s) === RUN TestMinimal_v1_26 I1102 00:58:33.390517 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:33.390756 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:33.390813 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:33.390857 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:33.390909 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:33.390996 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:33.391029 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:33.391067 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:33.391095 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:33.411090 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:33.412980 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:33.413044 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.705573 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:34.878568 34543 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1102 00:58:34.879814 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:34.880376 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:34.880388 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:34.881598 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:34.881645 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:34.883084 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:34.885598 34543 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1102 00:58:34.891379 34543 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1102 00:58:34.891873 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.892131 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.893317 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.894740 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.894871 34543 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1102 00:58:34.895201 34543 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1102 00:58:34.895375 34543 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1102 00:58:34.949158 34543 target.go:112] Terraform output is in /tmp/test1324453906/out I1102 00:58:34.953382 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:34.953634 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:34.953694 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:34.953744 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:34.953802 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:34.953846 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:34.953878 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:34.953915 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:34.953944 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:34.977416 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:34.980429 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:34.980531 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:35.018663 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:35.118672 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:35.119406 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:35.119502 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1102 00:58:35.119542 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:35.119536 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:35.119651 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:35.119558 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:35.120772 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:35.119715 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } W1102 00:58:35.121772 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:35.126422 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:35.133149 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:35.133666 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:35.133872 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:35.134920 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:35.136049 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:35.136191 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:35.136455 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:35.136631 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:35.139695 34543 target.go:168] Cloudformation output is in /tmp/test1032892008/out --- PASS: TestMinimal_v1_26 (1.76s) === RUN TestHetzner I1102 00:58:35.149400 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:36.175487 34543 executor.go:111] Tasks: 0 done / 50 total; 39 can run I1102 00:58:36.179237 34543 executor.go:111] Tasks: 39 done / 50 total; 5 can run I1102 00:58:36.180587 34543 executor.go:111] Tasks: 44 done / 50 total; 2 can run I1102 00:58:36.188869 34543 executor.go:111] Tasks: 46 done / 50 total; 4 can run I1102 00:58:36.191717 34543 executor.go:111] Tasks: 50 done / 50 total; 0 can run I1102 00:58:36.209177 34543 target.go:112] Terraform output is in /tmp/test2089913264/out --- PASS: TestHetzner (1.06s) === RUN TestNvidia I1102 00:58:36.212901 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:36.213140 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:36.213199 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:36.213300 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:36.213354 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:36.213397 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:36.213430 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:36.213471 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:36.213500 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:36.240493 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:36.243701 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.243808 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.282377 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:36.389888 34543 executor.go:111] Tasks: 0 done / 88 total; 44 can run W1102 00:58:36.390291 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:36.390414 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:36.390456 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:36.390600 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:36.390748 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:36.390999 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:36.393395 34543 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1102 00:58:36.398919 34543 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1102 00:58:36.399521 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.399789 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.400725 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.402062 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.402188 34543 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1102 00:58:36.402490 34543 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1102 00:58:36.402759 34543 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1102 00:58:36.456903 34543 target.go:112] Terraform output is in /tmp/test2666208784/out I1102 00:58:36.460973 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:36.461234 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:36.461312 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:36.461398 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:36.461487 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:36.461551 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:36.461609 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:36.461681 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:36.461731 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:36.487955 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:36.492436 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.492548 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.531502 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:36.632848 34543 executor.go:111] Tasks: 0 done / 88 total; 44 can run I1102 00:58:36.633125 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:36.633169 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:36.633282 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:36.633388 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:36.633506 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:36.633649 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:36.633813 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:36.633850 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:36.633997 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1102 00:58:36.635535 34543 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1102 00:58:36.640537 34543 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1102 00:58:36.640837 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.641105 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.642901 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.644417 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.644608 34543 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1102 00:58:36.644929 34543 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1102 00:58:36.645145 34543 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1102 00:58:36.649006 34543 target.go:168] Cloudformation output is in /tmp/test685692804/out --- PASS: TestNvidia (0.45s) === RUN TestMinimalGossip I1102 00:58:36.661971 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:36.662227 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:36.662294 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:36.662375 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:36.662435 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:36.662551 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:36.662598 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:36.662640 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:36.662680 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:36.700393 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.700542 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.841229 34543 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1102 00:58:36.841499 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:36.842260 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:36.842327 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:36.844560 34543 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1102 00:58:36.852016 34543 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1102 00:58:36.852469 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.852702 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.854795 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.859057 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.859189 34543 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1102 00:58:36.859546 34543 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1102 00:58:36.859852 34543 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1102 00:58:36.905138 34543 target.go:112] Terraform output is in /tmp/test1634102364/out --- PASS: TestMinimalGossip (0.25s) === RUN TestMinimalGossipIRSA I1102 00:58:36.907465 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:36.907628 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:36.907672 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:36.907704 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:36.907741 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:36.907807 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:36.907824 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:36.907849 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:36.907869 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:36.932189 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:36.932306 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:37.050619 34543 executor.go:111] Tasks: 0 done / 89 total; 44 can run W1102 00:58:37.051333 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:37.051545 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:37.051653 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:37.053260 34543 executor.go:111] Tasks: 44 done / 89 total; 20 can run I1102 00:58:37.056962 34543 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1102 00:58:37.057190 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:37.057477 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:37.057939 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:37.058820 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:37.058886 34543 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1102 00:58:37.059892 34543 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1102 00:58:37.060108 34543 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1102 00:58:37.099365 34543 target.go:112] Terraform output is in /tmp/test1516643220/out --- PASS: TestMinimalGossipIRSA (0.20s) === RUN TestMinimalGCE I1102 00:58:37.103841 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:37.105756 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1102 00:58:37.968202 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:37.968281 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:37.968380 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:58:37.968434 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1102 00:58:37.968456 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1102 00:58:37.968474 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:58:37.968535 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:58:37.969979 34543 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1102 00:58:37.977093 34543 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1102 00:58:37.983208 34543 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1102 00:58:37.986089 34543 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1102 00:58:37.986676 34543 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1102 00:58:38.017179 34543 target.go:112] Terraform output is in /tmp/test380363764/out --- PASS: TestMinimalGCE (0.92s) === RUN TestMinimalGCEPrivate I1102 00:58:38.021314 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:38.023216 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:58:38.812036 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:38.812127 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:38.812259 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W1102 00:58:38.812299 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W1102 00:58:38.812326 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1102 00:58:38.812361 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1102 00:58:38.812425 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:58:38.814271 34543 executor.go:111] Tasks: 0 done / 66 total; 40 can run I1102 00:58:38.821815 34543 executor.go:111] Tasks: 40 done / 66 total; 18 can run I1102 00:58:38.827482 34543 executor.go:111] Tasks: 58 done / 66 total; 4 can run I1102 00:58:38.830485 34543 executor.go:111] Tasks: 62 done / 66 total; 3 can run I1102 00:58:38.831943 34543 executor.go:111] Tasks: 65 done / 66 total; 1 can run I1102 00:58:38.832149 34543 executor.go:111] Tasks: 66 done / 66 total; 0 can run I1102 00:58:38.863691 34543 target.go:112] Terraform output is in /tmp/test1441753271/out --- PASS: TestMinimalGCEPrivate (0.85s) === RUN TestMinimalGCEInternalLoadBalancer I1102 00:58:38.867457 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:38.869479 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:58:39.046436 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:39.046520 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:39.046660 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W1102 00:58:39.046700 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W1102 00:58:39.046727 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1102 00:58:39.046762 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1102 00:58:39.046837 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1102 00:58:39.046904 34543 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1102 00:58:39.048721 34543 executor.go:111] Tasks: 0 done / 68 total; 41 can run I1102 00:58:39.057269 34543 executor.go:111] Tasks: 41 done / 68 total; 17 can run W1102 00:58:39.062805 34543 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1102 00:58:39.062848 34543 executor.go:111] Tasks: 58 done / 68 total; 4 can run I1102 00:58:39.065368 34543 executor.go:111] Tasks: 62 done / 68 total; 3 can run I1102 00:58:39.067299 34543 executor.go:111] Tasks: 65 done / 68 total; 2 can run I1102 00:58:39.067734 34543 executor.go:111] Tasks: 67 done / 68 total; 1 can run I1102 00:58:39.067991 34543 executor.go:111] Tasks: 68 done / 68 total; 0 can run I1102 00:58:39.098116 34543 target.go:112] Terraform output is in /tmp/test1766827591/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.23s) === RUN TestMinimalGCELongClusterName I1102 00:58:39.101774 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:39.103682 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1102 00:58:39.335211 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:39.335324 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:39.335461 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1102 00:58:39.335500 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1102 00:58:39.335527 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1102 00:58:39.335563 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1102 00:58:39.335639 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:58:39.337419 34543 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1102 00:58:39.342126 34543 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1102 00:58:39.349471 34543 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1102 00:58:39.352559 34543 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1102 00:58:39.352775 34543 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1102 00:58:39.384240 34543 target.go:112] Terraform output is in /tmp/test1902277297/out --- PASS: TestMinimalGCELongClusterName (0.29s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I1102 00:58:39.388028 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:39.389974 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1102 00:58:39.625783 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:39.625857 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:39.625985 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1102 00:58:39.626017 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1102 00:58:39.626040 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1102 00:58:39.626068 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1102 00:58:39.626126 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1102 00:58:39.626169 34543 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1102 00:58:39.631157 34543 executor.go:111] Tasks: 0 done / 67 total; 42 can run I1102 00:58:39.634826 34543 executor.go:111] Tasks: 42 done / 67 total; 16 can run W1102 00:58:39.640050 34543 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1102 00:58:39.640093 34543 executor.go:111] Tasks: 58 done / 67 total; 5 can run I1102 00:58:39.642880 34543 executor.go:111] Tasks: 63 done / 67 total; 2 can run I1102 00:58:39.643215 34543 executor.go:111] Tasks: 65 done / 67 total; 1 can run I1102 00:58:39.643498 34543 executor.go:111] Tasks: 66 done / 67 total; 1 can run I1102 00:58:39.643766 34543 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1102 00:58:39.676682 34543 target.go:112] Terraform output is in /tmp/test1304571738/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.29s) === RUN TestHA I1102 00:58:39.681330 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:39.681591 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:39.681655 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:39.681707 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:39.681762 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:39.681859 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:39.681891 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:39.681929 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:39.681957 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:39.716041 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:39.719556 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.719655 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.719712 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.719767 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.757802 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:39.871093 34543 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1102 00:58:39.871701 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:39.871953 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:39.871959 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:39.872027 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:39.872088 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:39.872015 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:39.872153 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:39.872361 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:39.874904 34543 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1102 00:58:39.883378 34543 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1102 00:58:39.883889 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.884042 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.884324 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.884631 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.888154 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.890813 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.891150 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.891580 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:39.891720 34543 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1102 00:58:39.892221 34543 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1102 00:58:39.892483 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:58:39.968668 34543 target.go:112] Terraform output is in /tmp/test4063060438/out --- PASS: TestHA (0.29s) === RUN TestHighAvailabilityGCE I1102 00:58:39.973772 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:39.975878 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:58:40.172682 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:58:40.172789 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:58:40.172909 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W1102 00:58:40.172946 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W1102 00:58:40.172971 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1102 00:58:40.173004 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1102 00:58:40.173067 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1102 00:58:40.173151 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1102 00:58:40.173218 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:58:40.175440 34543 executor.go:111] Tasks: 0 done / 83 total; 48 can run I1102 00:58:40.178291 34543 executor.go:111] Tasks: 48 done / 83 total; 20 can run I1102 00:58:40.184901 34543 executor.go:111] Tasks: 68 done / 83 total; 7 can run I1102 00:58:40.192946 34543 executor.go:111] Tasks: 75 done / 83 total; 5 can run I1102 00:58:40.194242 34543 executor.go:111] Tasks: 80 done / 83 total; 3 can run I1102 00:58:40.194518 34543 executor.go:111] Tasks: 83 done / 83 total; 0 can run I1102 00:58:40.235224 34543 target.go:112] Terraform output is in /tmp/test3207173122/out --- PASS: TestHighAvailabilityGCE (0.27s) === RUN TestComplex I1102 00:58:40.239852 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:40.240092 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:40.240153 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:40.240297 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:40.240360 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:40.240415 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:40.240457 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:40.240497 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:40.240528 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:40.275180 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:40.279490 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:40.279599 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.587236 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:41.780462 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1102 00:58:41.780858 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:41.781041 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:41.781743 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:41.781788 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:41.781954 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:41.783278 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:58:41.792195 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:58:41.792576 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.793036 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.794270 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.795451 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.795580 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:58:41.796057 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:58:41.796267 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:58:41.868766 34543 target.go:112] Terraform output is in /tmp/test660382846/out I1102 00:58:41.873042 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:41.873270 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:41.873335 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:41.873388 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:41.873444 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:41.873492 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:41.873525 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:41.873589 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:41.873620 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:41.909364 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:41.913642 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.913744 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:41.952714 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:42.144240 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:58:42.144905 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:42.144948 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:42.145084 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:42.145231 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:42.146899 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:42.151752 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:58:42.158245 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:58:42.158753 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.158973 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.160190 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.162005 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.162184 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:58:42.162537 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:58:42.162799 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:58:42.167349 34543 target.go:168] Cloudformation output is in /tmp/test457007570/out I1102 00:58:42.180489 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:42.180747 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:42.180819 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:42.180893 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:42.180980 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:42.181040 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:42.181092 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:42.181161 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:42.181209 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:42.217447 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:42.221923 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.222030 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.260635 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:42.454537 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:58:42.455464 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:42.455509 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1102 00:58:42.455621 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.455645 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:42.455738 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.458015 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:58:42.467712 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:58:42.468482 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.468739 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.469795 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.471917 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.472039 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:58:42.472451 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:58:42.472640 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:58:42.549095 34543 target.go:112] Terraform output is in /tmp/test3730035385/out --- PASS: TestComplex (2.31s) === RUN TestCompress I1102 00:58:42.553441 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:42.554102 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:42.554170 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:42.554222 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:42.554277 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:42.554374 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:42.554406 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:42.554444 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:42.554475 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:42.578138 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:42.581357 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.581460 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.620547 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:42.720504 34543 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1102 00:58:42.721002 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.721270 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:42.721308 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.721318 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:42.721459 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:42.723889 34543 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1102 00:58:42.731319 34543 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1102 00:58:42.731725 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.731881 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.740987 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.741903 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.742053 34543 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1102 00:58:42.742365 34543 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1102 00:58:42.742546 34543 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1102 00:58:42.789175 34543 target.go:112] Terraform output is in /tmp/test2386822336/out --- PASS: TestCompress (0.24s) === RUN TestExternalPolicies I1102 00:58:42.793487 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:42.793754 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:42.793815 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:42.793865 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:42.793927 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:42.794041 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:42.794074 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:42.794111 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:42.794140 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:42.827007 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:42.830603 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.830711 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.870176 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:42.971829 34543 executor.go:111] Tasks: 0 done / 100 total; 45 can run W1102 00:58:42.972409 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.972674 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:42.972792 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:42.972873 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:42.972831 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:42.972961 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:42.974587 34543 executor.go:111] Tasks: 45 done / 100 total; 20 can run I1102 00:58:42.979854 34543 executor.go:111] Tasks: 65 done / 100 total; 30 can run I1102 00:58:42.980182 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.980502 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.981261 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.982855 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:42.982977 34543 executor.go:111] Tasks: 95 done / 100 total; 3 can run I1102 00:58:42.983399 34543 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1102 00:58:42.983635 34543 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1102 00:58:43.051302 34543 target.go:112] Terraform output is in /tmp/test2247281616/out --- PASS: TestExternalPolicies (0.26s) === RUN TestMinimalIPv6 I1102 00:58:43.055635 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:43.055880 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:43.055943 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:43.056053 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:43.056107 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:43.056155 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:43.056184 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:43.056223 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:43.056254 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:43.081706 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:43.085420 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.085521 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.123999 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:43.294027 34543 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1102 00:58:43.294591 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:43.294710 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:43.294854 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:43.294890 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:43.294996 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:43.294982 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:43.297390 34543 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1102 00:58:43.302845 34543 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1102 00:58:43.303215 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.303412 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.306355 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.309908 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.310085 34543 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1102 00:58:43.310548 34543 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1102 00:58:43.310842 34543 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1102 00:58:43.372626 34543 target.go:112] Terraform output is in /tmp/test3553486382/out I1102 00:58:43.377194 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:43.377422 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:43.377477 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:43.377527 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:43.377580 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:43.377625 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:43.377653 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:43.377690 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:43.377721 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:43.402844 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:43.406503 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.406595 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.445503 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:43.618237 34543 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1102 00:58:43.618753 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:43.618789 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:43.618891 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:43.618903 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:43.619207 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:43.619329 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1102 00:58:43.619351 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:43.619348 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:43.619434 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:58:43.621858 34543 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1102 00:58:43.627644 34543 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1102 00:58:43.628007 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.629204 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.630192 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.634329 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.634550 34543 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1102 00:58:43.635005 34543 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1102 00:58:43.635376 34543 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1102 00:58:43.639403 34543 target.go:168] Cloudformation output is in /tmp/test2812073898/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I1102 00:58:43.650735 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:43.650972 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:43.651026 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:43.651076 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:43.651134 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:43.651221 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:43.651251 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:43.651288 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:43.651317 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:43.680493 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1102 00:58:43.683811 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.683893 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.723080 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:43.901966 34543 executor.go:111] Tasks: 0 done / 124 total; 46 can run W1102 00:58:43.902391 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:43.902763 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:43.903201 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:43.903268 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:43.903320 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:43.903438 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:43.906602 34543 executor.go:111] Tasks: 46 done / 124 total; 23 can run I1102 00:58:43.917182 34543 executor.go:111] Tasks: 69 done / 124 total; 36 can run I1102 00:58:43.917603 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.918095 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.918942 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.921299 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:43.921431 34543 executor.go:111] Tasks: 105 done / 124 total; 11 can run I1102 00:58:43.921984 34543 executor.go:111] Tasks: 116 done / 124 total; 8 can run I1102 00:58:43.922399 34543 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1102 00:58:43.999967 34543 target.go:112] Terraform output is in /tmp/test4014760223/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I1102 00:58:44.004568 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:44.004819 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:44.004880 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:44.004991 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:44.005051 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:44.005098 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:44.005128 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:44.005165 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:44.005196 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:44.031076 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:44.034654 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.034746 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.074080 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:44.534224 34543 executor.go:111] Tasks: 0 done / 109 total; 47 can run W1102 00:58:44.534875 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:44.535155 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:44.535185 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:44.535210 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:44.535250 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:44.535319 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:44.537667 34543 executor.go:111] Tasks: 47 done / 109 total; 21 can run I1102 00:58:44.542967 34543 executor.go:111] Tasks: 68 done / 109 total; 31 can run I1102 00:58:44.543368 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.543727 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.546198 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.549123 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.549272 34543 executor.go:111] Tasks: 99 done / 109 total; 5 can run I1102 00:58:44.549822 34543 executor.go:111] Tasks: 104 done / 109 total; 5 can run I1102 00:58:44.550145 34543 executor.go:111] Tasks: 109 done / 109 total; 0 can run I1102 00:58:44.615678 34543 target.go:112] Terraform output is in /tmp/test3557346746/out --- PASS: TestMinimalIPv6Calico (0.62s) === RUN TestMinimalIPv6Cilium I1102 00:58:44.621179 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:44.621453 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:44.621517 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:44.621630 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:44.621685 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:44.621736 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:44.621766 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:44.621807 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:44.621837 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:44.648031 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:44.651867 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.651969 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.691523 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:44.900730 34543 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1102 00:58:44.902099 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:44.902802 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:44.903389 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:44.904136 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:44.905816 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:44.905967 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:44.908241 34543 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1102 00:58:44.913864 34543 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1102 00:58:44.914114 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.914273 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.915687 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.917551 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:44.917701 34543 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1102 00:58:44.918270 34543 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1102 00:58:44.918829 34543 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1102 00:58:44.985035 34543 target.go:112] Terraform output is in /tmp/test1899758754/out I1102 00:58:44.989410 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:44.989634 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:44.989692 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:44.989743 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:44.989797 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:44.989841 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:44.989869 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:44.989906 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:44.989939 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:45.015646 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:45.019343 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.019436 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.058862 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:45.271144 34543 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1102 00:58:45.271328 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:45.271395 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:45.271619 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:45.271703 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:45.271713 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:45.271785 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:45.271786 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:58:45.271812 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:45.271896 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:45.278961 34543 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1102 00:58:45.284631 34543 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1102 00:58:45.285101 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.285237 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.286370 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.288017 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.288194 34543 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1102 00:58:45.288771 34543 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1102 00:58:45.289236 34543 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1102 00:58:45.293238 34543 target.go:168] Cloudformation output is in /tmp/test1724435871/out --- PASS: TestMinimalIPv6Cilium (0.68s) === RUN TestMinimalWarmPool I1102 00:58:45.305042 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:45.305279 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:45.305336 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:45.305383 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:45.305439 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:45.305523 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:45.305553 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:45.305591 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:45.305620 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:45.335159 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:45.338742 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.338844 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.377828 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:45.580138 34543 executor.go:111] Tasks: 0 done / 89 total; 45 can run W1102 00:58:45.580624 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:45.580635 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:45.581339 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:45.581537 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:45.581581 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:45.581715 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:45.585097 34543 executor.go:111] Tasks: 45 done / 89 total; 19 can run I1102 00:58:45.593014 34543 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1102 00:58:45.593511 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.593779 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.595051 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.596546 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.596665 34543 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1102 00:58:45.597026 34543 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1102 00:58:45.597309 34543 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1102 00:58:45.652059 34543 target.go:112] Terraform output is in /tmp/test949003224/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I1102 00:58:45.656395 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:45.656644 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:45.656705 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:45.656814 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:45.656888 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:45.656940 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:45.656974 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:45.657012 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:45.657044 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:45.683929 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:45.687293 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.687390 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.727417 34543 zones.go:93] ListHostedZonesPages { } W1102 00:58:45.816662 34543 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W1102 00:58:45.816724 34543 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W1102 00:58:45.818666 34543 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:45.831705 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:58:45.832224 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:45.832340 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:45.832680 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:45.832709 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:45.832705 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:45.832874 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:45.832894 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:45.832918 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:45.833053 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I1102 00:58:45.835292 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:45.840643 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:45.840858 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.841253 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.842436 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.843745 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.843887 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:45.844174 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:45.844380 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:45.847556 34543 target.go:168] Cloudformation output is in /tmp/test3056738923/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I1102 00:58:45.860962 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:45.861247 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:45.861305 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:45.861356 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:45.861413 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:45.861497 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:45.861528 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:45.861566 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:45.861597 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:45.887341 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:45.890481 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.890580 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:45.929774 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:46.033510 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:58:46.033819 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:46.034134 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:46.034551 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:46.034583 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:46.034617 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:46.034669 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:46.036900 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:46.041934 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:46.042378 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.042628 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.043715 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.044870 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.044984 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:46.045292 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:46.045482 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:46.098387 34543 target.go:112] Terraform output is in /tmp/test4199773480/out I1102 00:58:46.102143 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:46.102379 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:46.102440 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:46.102495 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:46.102551 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:46.102595 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:46.102627 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:46.102669 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:46.102699 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:46.127822 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:46.130975 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.131074 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.171014 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:46.274160 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:46.274502 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:46.274529 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:46.274626 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:46.274683 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:46.274855 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:46.274990 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1102 00:58:46.275033 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:46.275017 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:46.275146 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:58:46.277584 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:46.283173 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:46.283366 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.283654 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.284561 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.286008 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.286136 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:46.286391 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:46.286576 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:46.289507 34543 target.go:168] Cloudformation output is in /tmp/test3937967782/out --- PASS: TestMinimalGp3 (0.44s) === RUN TestMinimalLongClusterName I1102 00:58:46.299296 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:46.299525 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:46.299578 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:46.299624 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:46.299673 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:46.299751 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:46.299785 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:46.299822 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:46.299852 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:46.329245 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:46.332855 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.332955 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:46.370708 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:46.476994 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:58:46.477451 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:46.477572 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:46.477959 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:46.478224 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:46.478269 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:46.478422 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:46.480790 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:46.486260 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:46.486523 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.486852 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.487962 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.488924 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.489042 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:46.489350 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:46.489551 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:46.544718 34543 target.go:112] Terraform output is in /tmp/test4225668145/out I1102 00:58:46.548860 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:46.549083 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:46.549144 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:46.549194 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:46.549248 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:46.549293 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:46.549323 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:46.549361 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:46.549390 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:46.573526 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:46.576640 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.576741 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:58:46.615417 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:46.714279 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:58:46.714917 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:46.714951 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:46.715092 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:46.715054 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:46.715293 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:46.715446 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:46.715461 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:46.715497 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:46.715630 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } I1102 00:58:46.717689 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:58:46.722938 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:58:46.723377 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.723554 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.725027 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.728024 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.728495 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:58:46.730135 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:58:46.730386 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:58:46.733634 34543 target.go:168] Cloudformation output is in /tmp/test3741935076/out --- PASS: TestMinimalLongClusterName (0.45s) === RUN TestExistingIAMCloudformation I1102 00:58:46.744504 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:46.744753 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:46.744805 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:46.744899 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:46.744954 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:46.744998 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:46.745026 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:46.745062 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:46.745090 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:46.767750 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:46.770143 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.770253 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.808845 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:46.911384 34543 executor.go:111] Tasks: 0 done / 77 total; 41 can run W1102 00:58:46.911865 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:46.912129 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:46.912322 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1102 00:58:46.912176 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:46.912361 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:46.912393 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:46.912383 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:46.912492 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:46.912524 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:58:46.914156 34543 executor.go:111] Tasks: 41 done / 77 total; 11 can run I1102 00:58:46.919246 34543 executor.go:111] Tasks: 52 done / 77 total; 21 can run I1102 00:58:46.919463 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.919744 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.920881 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.921724 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.921864 34543 executor.go:111] Tasks: 73 done / 77 total; 2 can run I1102 00:58:46.922132 34543 executor.go:111] Tasks: 75 done / 77 total; 2 can run I1102 00:58:46.922313 34543 executor.go:111] Tasks: 77 done / 77 total; 0 can run I1102 00:58:46.924508 34543 target.go:168] Cloudformation output is in /tmp/test1606220407/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I1102 00:58:46.933746 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:46.934078 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:46.934184 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:46.934285 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:46.934402 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:46.934558 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:46.934624 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:46.934706 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:46.934758 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:46.967648 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:46.970365 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.970436 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.970483 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:46.970529 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.008586 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:47.121840 34543 executor.go:111] Tasks: 0 done / 146 total; 53 can run W1102 00:58:47.122339 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:47.122545 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.122610 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:47.122644 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:47.122793 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:47.122797 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:47.122900 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.122911 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:47.125455 34543 executor.go:111] Tasks: 53 done / 146 total; 26 can run I1102 00:58:47.134875 34543 executor.go:111] Tasks: 79 done / 146 total; 58 can run I1102 00:58:47.135282 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.135742 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.135962 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.136205 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.136493 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.138143 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.138330 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.138511 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.138636 34543 executor.go:111] Tasks: 137 done / 146 total; 5 can run I1102 00:58:47.139094 34543 executor.go:111] Tasks: 142 done / 146 total; 4 can run I1102 00:58:47.139397 34543 executor.go:111] Tasks: 146 done / 146 total; 0 can run I1102 00:58:47.236189 34543 target.go:112] Terraform output is in /tmp/test48623836/out --- PASS: TestExistingSG (0.31s) === RUN TestBastionAdditionalUserData I1102 00:58:47.241489 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:47.241762 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:47.241825 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:47.241880 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:47.241936 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:47.242032 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:47.242062 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:47.242101 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:47.242134 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:47.276269 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:47.279713 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.279819 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.279884 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.318208 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:47.420932 34543 executor.go:111] Tasks: 0 done / 122 total; 48 can run W1102 00:58:47.421571 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.421664 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:47.421771 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:47.421818 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1102 00:58:47.421956 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.421987 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:47.422327 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.424239 34543 executor.go:111] Tasks: 48 done / 122 total; 29 can run I1102 00:58:47.431218 34543 executor.go:111] Tasks: 77 done / 122 total; 36 can run I1102 00:58:47.431531 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.431848 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.432102 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.434805 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.435164 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.438059 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.438220 34543 executor.go:111] Tasks: 113 done / 122 total; 5 can run I1102 00:58:47.438822 34543 executor.go:111] Tasks: 118 done / 122 total; 4 can run I1102 00:58:47.439151 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:58:47.515157 34543 target.go:112] Terraform output is in /tmp/test2599006791/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I1102 00:58:47.519671 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:47.519965 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:47.520028 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:47.520091 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:47.520159 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:47.520290 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:47.520325 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:47.520377 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:47.520411 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:47.548386 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:47.551760 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.551856 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.551937 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.589821 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:47.717330 34543 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1102 00:58:47.717728 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:47.717962 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:47.718136 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:47.718468 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:47.718570 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:47.718512 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:47.718647 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:47.720778 34543 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1102 00:58:47.726285 34543 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1102 00:58:47.726764 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.727006 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.727160 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.727236 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.728463 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.730053 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.730188 34543 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1102 00:58:47.730745 34543 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1102 00:58:47.731031 34543 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1102 00:58:47.809745 34543 target.go:112] Terraform output is in /tmp/test265587391/out --- PASS: TestPrivateWeave (0.29s) === RUN TestPrivateFlannel I1102 00:58:47.814242 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:47.814531 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:47.814594 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:47.814720 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:47.814799 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:47.814851 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:47.814897 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:47.814946 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:47.814979 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:47.842962 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:47.846571 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.846672 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.846759 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:47.885267 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:48.075014 34543 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1102 00:58:48.075254 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:48.075317 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:48.076174 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:48.076195 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:48.078870 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:48.076237 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:48.079044 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:48.082909 34543 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1102 00:58:48.089420 34543 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1102 00:58:48.089953 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.090142 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.090237 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.090467 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.091557 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.093013 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.093142 34543 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1102 00:58:48.093638 34543 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1102 00:58:48.093957 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:58:48.170470 34543 target.go:112] Terraform output is in /tmp/test3726628909/out --- PASS: TestPrivateFlannel (0.36s) === RUN TestPrivateCalico I1102 00:58:48.175428 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:48.175674 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:48.175737 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:48.175868 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:48.175927 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:48.175971 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:48.176001 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:48.176038 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:48.176068 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:48.209563 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:48.213938 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.214105 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.214221 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.252764 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:48.708732 34543 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1102 00:58:48.709074 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:48.709245 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:48.709289 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:48.709269 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:48.709357 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:48.709399 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:48.709476 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:48.715910 34543 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1102 00:58:48.722031 34543 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1102 00:58:48.722518 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.722840 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.723076 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.723291 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.724149 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.725503 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.725650 34543 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1102 00:58:48.726052 34543 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1102 00:58:48.726338 34543 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1102 00:58:48.811746 34543 target.go:112] Terraform output is in /tmp/test1204776977/out --- PASS: TestPrivateCalico (0.64s) === RUN TestPrivateCilium I1102 00:58:48.817435 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:48.817727 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:48.817800 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:48.817941 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:48.818009 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:48.818074 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:48.818109 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:48.818163 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:48.818197 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:48.846252 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:48.848948 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.849026 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.849072 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:48.886207 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:49.035540 34543 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1102 00:58:49.035861 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:49.035941 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:49.036115 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:49.037252 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:49.037306 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:49.037453 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:49.039508 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:49.043204 34543 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1102 00:58:49.048889 34543 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1102 00:58:49.049674 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.049890 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.050108 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.050314 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.051049 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.052088 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.052288 34543 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1102 00:58:49.053005 34543 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1102 00:58:49.053231 34543 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1102 00:58:49.131596 34543 target.go:112] Terraform output is in /tmp/test4065395674/out I1102 00:58:49.135903 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:49.136159 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:49.136238 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:49.136320 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:49.136419 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:49.136485 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:49.136545 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:49.136619 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:49.136670 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:49.165385 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:49.169053 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.169164 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.169254 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.207437 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:49.357039 34543 executor.go:111] Tasks: 0 done / 120 total; 47 can run I1102 00:58:49.357499 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:49.357611 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:49.357663 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:49.357689 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:49.357980 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1102 00:58:49.358000 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:49.358016 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:49.358116 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:49.358121 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:49.357871 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1102 00:58:49.360646 34543 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1102 00:58:49.370477 34543 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1102 00:58:49.370777 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.370938 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.371045 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.371262 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.372147 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.374064 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.374217 34543 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1102 00:58:49.374681 34543 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1102 00:58:49.374950 34543 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1102 00:58:49.379566 34543 target.go:168] Cloudformation output is in /tmp/test1425731536/out --- PASS: TestPrivateCilium (0.57s) === RUN TestPrivateCilium2 I1102 00:58:49.392380 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:49.392617 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:49.392668 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:49.392755 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:49.392810 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:49.392854 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:49.392884 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:49.392925 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:49.392953 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:49.424636 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:49.428491 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.428589 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.428655 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:49.467774 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:50.107847 34543 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1102 00:58:50.108113 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:50.108252 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:50.110637 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:50.110728 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:50.110769 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:50.110888 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:50.110996 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:50.116814 34543 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1102 00:58:50.122465 34543 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1102 00:58:50.123140 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.123405 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.123748 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.123878 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.125112 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.126994 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.127129 34543 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1102 00:58:50.127668 34543 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1102 00:58:50.127964 34543 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1102 00:58:50.205517 34543 target.go:112] Terraform output is in /tmp/test2896763067/out I1102 00:58:50.213097 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:50.213330 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:50.213391 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:50.213446 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:50.213504 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:50.213565 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:50.213596 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:50.213645 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:50.213676 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:50.245313 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:50.249266 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.249439 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.249555 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.287948 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:50.943786 34543 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1102 00:58:50.944407 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:50.944506 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1102 00:58:50.944535 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:50.944552 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:50.944704 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:58:50.944475 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:50.944862 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1102 00:58:50.944943 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:50.944893 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:50.945043 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1102 00:58:50.952469 34543 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1102 00:58:50.958305 34543 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1102 00:58:50.958514 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.958928 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.959085 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.959157 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.960503 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.961232 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:50.961374 34543 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1102 00:58:50.961772 34543 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1102 00:58:50.962125 34543 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1102 00:58:50.966115 34543 target.go:168] Cloudformation output is in /tmp/test2989972401/out --- PASS: TestPrivateCilium2 (1.59s) === RUN TestPrivateCiliumAdvanced I1102 00:58:50.978869 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:50.979097 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:50.979150 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:50.979240 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:50.979292 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:50.979336 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:50.979368 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:50.979408 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:50.979437 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:51.017943 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:51.021557 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.021652 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.021720 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.059828 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:51.217803 34543 executor.go:111] Tasks: 0 done / 126 total; 53 can run W1102 00:58:51.218372 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:51.218567 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:51.218782 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:51.218806 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:51.218877 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:51.218913 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:51.219112 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:51.221361 34543 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1102 00:58:51.231684 34543 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1102 00:58:51.232272 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.232455 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.232722 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.232969 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.234000 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.234991 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.235122 34543 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1102 00:58:51.235575 34543 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1102 00:58:51.235904 34543 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1102 00:58:51.315910 34543 target.go:112] Terraform output is in /tmp/test3734515128/out I1102 00:58:51.320494 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:51.320719 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:51.320779 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:51.320833 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:51.320889 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:51.320934 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:51.320966 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:51.321005 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:51.321035 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:51.358386 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:51.362089 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.362223 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.362293 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.399879 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:51.551512 34543 executor.go:111] Tasks: 0 done / 126 total; 53 can run W1102 00:58:51.551725 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:51.551814 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:51.551840 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:51.551934 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:51.552008 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:51.552347 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:51.553315 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:51.553462 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:58:51.553477 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:51.553587 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I1102 00:58:51.560420 34543 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1102 00:58:51.567575 34543 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1102 00:58:51.567776 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.568242 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.568480 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.568612 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.568975 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.570855 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.571013 34543 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1102 00:58:51.571399 34543 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1102 00:58:51.571648 34543 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1102 00:58:51.575915 34543 target.go:168] Cloudformation output is in /tmp/test4121485673/out --- PASS: TestPrivateCiliumAdvanced (0.61s) === RUN TestPrivateCanal I1102 00:58:51.592737 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:51.593148 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:51.593237 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:51.593426 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:51.593499 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:51.593563 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:51.593598 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:51.593635 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:51.593665 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:51.621685 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:51.625135 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.625226 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.625290 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:51.663499 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:52.137074 34543 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1102 00:58:52.137647 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.137933 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:52.137993 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:52.138025 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:52.138126 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:52.138166 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:52.138220 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.140340 34543 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1102 00:58:52.146620 34543 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1102 00:58:52.146997 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.147332 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.147481 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.147781 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.149290 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.154311 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:52.154464 34543 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1102 00:58:52.154970 34543 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1102 00:58:52.155248 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:58:52.229654 34543 target.go:112] Terraform output is in /tmp/test3186221914/out --- PASS: TestPrivateCanal (0.64s) === RUN TestPrivateKopeio I1102 00:58:52.235129 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:52.235381 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:52.235445 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:52.235497 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:52.235552 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:52.235652 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:52.235683 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:52.235720 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:52.235752 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:52.265447 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:52.268644 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.268733 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.268788 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.307037 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:52.420803 34543 executor.go:111] Tasks: 0 done / 126 total; 47 can run W1102 00:58:52.421421 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:52.421440 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.421647 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:52.421777 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.421696 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:52.421834 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:52.423422 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:52.427931 34543 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1102 00:58:52.434178 34543 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1102 00:58:52.434752 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.435024 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.435359 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.435639 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.436491 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.437179 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.437304 34543 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1102 00:58:52.437911 34543 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1102 00:58:52.438270 34543 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1102 00:58:52.518287 34543 target.go:112] Terraform output is in /tmp/test1057286558/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I1102 00:58:52.522543 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:52.522792 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:52.522852 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:52.522903 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:52.522960 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:52.523054 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:52.523084 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:52.523123 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:52.523153 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:52.552947 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:52.556964 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.557053 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.557130 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.595692 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:52.697420 34543 executor.go:111] Tasks: 0 done / 108 total; 45 can run W1102 00:58:52.697917 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.698056 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:52.698128 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:52.698292 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.698432 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:52.698468 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:52.698574 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:52.705664 34543 executor.go:111] Tasks: 45 done / 108 total; 25 can run I1102 00:58:52.711213 34543 executor.go:111] Tasks: 70 done / 108 total; 31 can run I1102 00:58:52.711412 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.711751 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.711915 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.712127 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.712992 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.714351 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.714488 34543 executor.go:111] Tasks: 101 done / 108 total; 4 can run I1102 00:58:52.715024 34543 executor.go:111] Tasks: 105 done / 108 total; 3 can run I1102 00:58:52.715320 34543 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1102 00:58:52.787201 34543 target.go:112] Terraform output is in /tmp/test1775996937/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I1102 00:58:52.791563 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:52.791819 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:52.791885 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:52.792002 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:52.792058 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:52.792106 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:52.792135 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:52.792172 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:52.792208 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:52.821237 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:52.824598 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.824692 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.824755 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.863493 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:52.967315 34543 executor.go:111] Tasks: 0 done / 107 total; 45 can run W1102 00:58:52.967737 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:52.967857 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.968175 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:52.968212 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:52.968306 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:52.968445 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:52.968459 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:52.970552 34543 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1102 00:58:52.971124 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:52.979473 34543 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1102 00:58:52.979798 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.980134 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.980421 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.980612 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.980991 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.982735 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:52.982954 34543 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1102 00:58:52.983475 34543 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1102 00:58:52.983731 34543 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1102 00:58:53.051745 34543 target.go:112] Terraform output is in /tmp/test2959067083/out --- PASS: TestPrivateSharedSubnet (0.26s) === RUN TestPrivateSharedIP I1102 00:58:53.056405 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:53.056645 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:53.056700 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:53.056805 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:53.056861 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:53.056929 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:53.056960 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:53.056997 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:53.057026 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:53.085085 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:53.088314 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.088401 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.088462 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.126697 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:53.230248 34543 executor.go:111] Tasks: 0 done / 116 total; 45 can run W1102 00:58:53.230548 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:53.230573 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:53.230606 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:53.230705 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:53.230930 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:53.231145 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:53.231302 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:53.238138 34543 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1102 00:58:53.238869 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:53.244419 34543 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1102 00:58:53.244687 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.244943 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.245290 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.245562 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.245948 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.247677 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.247799 34543 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1102 00:58:53.248304 34543 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1102 00:58:53.248635 34543 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1102 00:58:53.322299 34543 target.go:112] Terraform output is in /tmp/test2147000349/out I1102 00:58:53.326581 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:53.326804 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:53.326887 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:53.326947 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:53.327003 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:53.327046 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:53.327077 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:53.327117 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:53.327147 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:53.354924 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:53.358355 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.358450 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.358513 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.396721 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:58:53.496541 34543 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1102 00:58:53.497371 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:58:53.497417 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:53.497564 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:58:53.497521 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:58:53.497709 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:53.497707 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:53.497813 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1102 00:58:53.497734 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:58:53.497829 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:53.497902 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1102 00:58:53.499786 34543 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1102 00:58:53.500361 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:53.504866 34543 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1102 00:58:53.505142 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.506444 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.506829 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.507337 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.509166 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.512148 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:58:53.512314 34543 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1102 00:58:53.512742 34543 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1102 00:58:53.513047 34543 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1102 00:58:53.517027 34543 target.go:168] Cloudformation output is in /tmp/test1532462513/out --- PASS: TestPrivateSharedIP (0.47s) === RUN TestPrivateDns1 I1102 00:58:53.530103 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:53.530351 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:53.530403 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:53.530452 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:53.530505 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:53.530585 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:53.530615 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:53.530651 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:53.530680 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:53.561945 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.562020 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.562070 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.716327 34543 executor.go:111] Tasks: 0 done / 120 total; 46 can run I1102 00:58:53.716859 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:53.718442 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:53.719913 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:53.720181 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:53.724513 34543 executor.go:111] Tasks: 46 done / 120 total; 24 can run I1102 00:58:53.724734 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I1102 00:58:53.724786 34543 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I1102 00:58:53.724916 34543 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I1102 00:58:53.724947 34543 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I1102 00:58:53.725164 34543 executor.go:111] Tasks: 70 done / 120 total; 35 can run I1102 00:58:53.725714 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.725954 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.729166 34543 executor.go:111] Tasks: 105 done / 120 total; 5 can run W1102 00:58:53.729241 34543 api.go:53] PageSize not implemented W1102 00:58:53.729444 34543 api.go:53] PageSize not implemented I1102 00:58:53.734336 34543 executor.go:111] Tasks: 110 done / 120 total; 6 can run I1102 00:58:53.734686 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.734964 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.735953 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.737337 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.737460 34543 executor.go:111] Tasks: 116 done / 120 total; 2 can run I1102 00:58:53.737835 34543 executor.go:111] Tasks: 118 done / 120 total; 2 can run I1102 00:58:53.738024 34543 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1102 00:58:53.821626 34543 target.go:112] Terraform output is in /tmp/test2362729615/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I1102 00:58:53.826146 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:53.826446 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:53.826517 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:53.826602 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:53.826688 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:53.826805 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:53.826861 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:53.826953 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:53.827002 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:53.857097 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.857201 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.857279 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.990405 34543 executor.go:111] Tasks: 0 done / 116 total; 44 can run W1102 00:58:53.990884 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:53.991084 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:53.991528 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:53.991682 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:53.993711 34543 executor.go:111] Tasks: 44 done / 116 total; 22 can run I1102 00:58:53.994107 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:58:53.994240 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I1102 00:58:53.994302 34543 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I1102 00:58:53.994401 34543 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I1102 00:58:53.994421 34543 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I1102 00:58:53.994475 34543 executor.go:111] Tasks: 66 done / 116 total; 35 can run I1102 00:58:53.995216 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.995389 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:53.998409 34543 executor.go:111] Tasks: 101 done / 116 total; 5 can run W1102 00:58:53.998517 34543 api.go:53] PageSize not implemented W1102 00:58:53.998650 34543 api.go:53] PageSize not implemented I1102 00:58:54.007069 34543 executor.go:111] Tasks: 106 done / 116 total; 6 can run I1102 00:58:54.007348 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.007643 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.008843 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.009851 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.009971 34543 executor.go:111] Tasks: 112 done / 116 total; 2 can run I1102 00:58:54.010327 34543 executor.go:111] Tasks: 114 done / 116 total; 2 can run I1102 00:58:54.010518 34543 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1102 00:58:54.083786 34543 target.go:112] Terraform output is in /tmp/test582070030/out --- PASS: TestPrivateDns2 (0.26s) === RUN TestDiscoveryFeatureGate I1102 00:58:54.088165 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:54.088449 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:54.088520 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:54.088648 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:54.088718 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:54.088774 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:54.088816 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:54.088860 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:54.088899 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:54.115653 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:54.123584 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.123748 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.162615 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:54.342727 34543 executor.go:111] Tasks: 0 done / 99 total; 51 can run I1102 00:58:54.343230 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:54.343262 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:54.343370 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:54.343486 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:54.343481 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:54.343796 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:54.346378 34543 executor.go:111] Tasks: 51 done / 99 total; 23 can run I1102 00:58:54.356975 34543 executor.go:111] Tasks: 74 done / 99 total; 21 can run I1102 00:58:54.357562 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.357797 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.359338 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.360758 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.360884 34543 executor.go:111] Tasks: 95 done / 99 total; 2 can run I1102 00:58:54.361194 34543 executor.go:111] Tasks: 97 done / 99 total; 2 can run I1102 00:58:54.361449 34543 executor.go:111] Tasks: 99 done / 99 total; 0 can run I1102 00:58:54.419641 34543 target.go:112] Terraform output is in /tmp/test84464632/out --- PASS: TestDiscoveryFeatureGate (0.34s) === RUN TestVFSServiceAccountIssuerDiscovery I1102 00:58:54.425024 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:54.425263 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:54.425321 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:54.425372 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:54.425427 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:54.425520 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:54.425550 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:54.425588 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:54.425620 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:54.450528 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:54.453658 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.453758 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.492687 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:54.593364 34543 executor.go:111] Tasks: 0 done / 90 total; 45 can run W1102 00:58:54.593680 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:54.593912 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:54.593955 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:54.594087 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:54.594334 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:54.594392 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:54.597019 34543 executor.go:111] Tasks: 45 done / 90 total; 20 can run I1102 00:58:54.602142 34543 executor.go:111] Tasks: 65 done / 90 total; 21 can run I1102 00:58:54.602418 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.602585 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.603802 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.604875 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.604993 34543 executor.go:111] Tasks: 86 done / 90 total; 2 can run I1102 00:58:54.605326 34543 executor.go:111] Tasks: 88 done / 90 total; 2 can run I1102 00:58:54.605594 34543 executor.go:111] Tasks: 90 done / 90 total; 0 can run I1102 00:58:54.661761 34543 target.go:112] Terraform output is in /tmp/test4158659135/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.24s) === RUN TestAWSLBController I1102 00:58:54.665943 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:54.666188 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:54.666246 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:54.666347 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:54.666407 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:54.666452 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:54.666482 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:54.666520 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:54.666548 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:54.694959 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:54.698230 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.698327 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:54.736175 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:55.332706 34543 executor.go:111] Tasks: 0 done / 96 total; 49 can run W1102 00:58:55.333343 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:55.333411 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:55.333479 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:55.333605 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:55.333559 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:55.333725 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:55.338874 34543 executor.go:111] Tasks: 49 done / 96 total; 22 can run I1102 00:58:55.346381 34543 executor.go:111] Tasks: 71 done / 96 total; 21 can run I1102 00:58:55.346716 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.347003 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.347879 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.349396 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.349518 34543 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1102 00:58:55.349838 34543 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1102 00:58:55.350062 34543 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1102 00:58:55.411541 34543 target.go:112] Terraform output is in /tmp/test1617030551/out --- PASS: TestAWSLBController (0.76s) === RUN TestManyAddons I1102 00:58:55.424419 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:55.424856 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:55.424967 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:55.425063 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:55.425163 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:55.425331 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:55.425392 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:55.425453 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:55.425483 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:55.453072 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:55.456814 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.456930 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:55.495191 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:56.279406 34543 executor.go:111] Tasks: 0 done / 95 total; 50 can run I1102 00:58:56.279958 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:58:56.280476 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:56.280510 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:56.280570 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:58:56.280579 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:58:56.280669 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:56.285894 34543 executor.go:111] Tasks: 50 done / 95 total; 19 can run I1102 00:58:56.293342 34543 executor.go:111] Tasks: 69 done / 95 total; 22 can run I1102 00:58:56.293907 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.294117 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.295049 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.296607 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.296729 34543 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1102 00:58:56.297074 34543 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1102 00:58:56.297268 34543 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1102 00:58:56.352894 34543 target.go:112] Terraform output is in /tmp/test3496906756/out --- PASS: TestManyAddons (0.94s) === RUN TestManyAddonsCCMIRSA I1102 00:58:56.364563 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:56.364833 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:56.364897 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:56.365012 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:56.365067 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:56.365113 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:56.365142 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:56.365180 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:56.365209 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:56.393029 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:58:56.396817 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.396936 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:56.435807 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:57.279052 34543 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1102 00:58:57.279901 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:57.280142 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:57.280155 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:58:57.280336 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:57.280195 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:57.280401 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:57.287717 34543 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1102 00:58:57.293721 34543 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1102 00:58:57.293985 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.294211 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.295159 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.296637 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.296765 34543 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1102 00:58:57.297100 34543 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1102 00:58:57.297296 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:58:57.367197 34543 target.go:112] Terraform output is in /tmp/test408003312/out --- PASS: TestManyAddonsCCMIRSA (1.01s) === RUN TestManyAddonsCCMIRSA23 I1102 00:58:57.377416 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:57.377669 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:57.377730 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:57.377841 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:57.377898 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:57.377948 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:57.377981 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:57.378020 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:57.378050 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:57.404905 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1102 00:58:57.408415 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.408503 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:57.446704 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:58.334148 34543 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1102 00:58:58.335211 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:58.335354 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:58.337367 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:58.337680 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:58.335395 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:58.338586 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:58.343883 34543 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1102 00:58:58.349703 34543 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1102 00:58:58.350049 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.350225 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.351534 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.353225 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.353362 34543 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1102 00:58:58.353907 34543 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1102 00:58:58.354264 34543 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1102 00:58:58.428463 34543 target.go:112] Terraform output is in /tmp/test2881609933/out --- PASS: TestManyAddonsCCMIRSA23 (1.06s) === RUN TestManyAddonsCCMIRSA24 I1102 00:58:58.438799 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:58.439049 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:58.439112 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:58.439216 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:58.439274 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:58.439319 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:58.439349 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:58.439390 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:58.439419 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:58.466024 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:58:58.469315 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.469396 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:58.507887 34543 zones.go:93] ListHostedZonesPages { } I1102 00:58:59.385760 34543 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1102 00:58:59.386325 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:59.386773 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:58:59.386901 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:58:59.386968 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:58:59.387012 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:58:59.387138 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:58:59.390520 34543 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1102 00:58:59.399463 34543 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1102 00:58:59.399671 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.399822 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.400893 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.402337 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.402472 34543 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1102 00:58:59.402814 34543 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1102 00:58:59.403043 34543 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1102 00:58:59.472743 34543 target.go:112] Terraform output is in /tmp/test4048673263/out --- PASS: TestManyAddonsCCMIRSA24 (1.04s) === RUN TestManyAddonsCCMIRSA25 I1102 00:58:59.482302 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:59.482591 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:59.482653 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:59.482780 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:59.482855 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:59.482914 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:59.482955 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:59.483007 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:59.483042 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:59.510552 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:58:59.514188 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.514282 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:58:59.554671 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:00.424893 34543 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1102 00:59:00.425524 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:00.425666 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:00.425726 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:00.425853 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:00.426070 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:00.426143 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:00.433799 34543 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1102 00:59:00.439324 34543 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1102 00:59:00.439533 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.439912 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.441115 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.442490 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.442623 34543 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1102 00:59:00.442925 34543 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1102 00:59:00.443130 34543 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1102 00:59:00.511113 34543 target.go:112] Terraform output is in /tmp/test3635667361/out --- PASS: TestManyAddonsCCMIRSA25 (1.04s) === RUN TestManyAddonsCCMIRSA26 I1102 00:59:00.521733 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:00.521986 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:00.522049 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:00.522102 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:00.522156 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:00.522272 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:00.522302 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:00.522340 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:00.522368 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:00.548819 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:59:00.552215 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.552301 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:00.591602 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:01.466387 34543 executor.go:111] Tasks: 0 done / 111 total; 59 can run W1102 00:59:01.466652 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:01.469543 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:01.470172 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:01.471937 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:01.471967 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:01.472078 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:01.475455 34543 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1102 00:59:01.485338 34543 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1102 00:59:01.485714 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.485931 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.487333 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.488415 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.488565 34543 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1102 00:59:01.488904 34543 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1102 00:59:01.489107 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:59:01.556722 34543 target.go:112] Terraform output is in /tmp/test730589472/out --- PASS: TestManyAddonsCCMIRSA26 (1.05s) === RUN TestCCM I1102 00:59:01.567839 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:01.568077 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:01.568138 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:01.568191 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:01.568245 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:01.568333 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:01.568363 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:01.568399 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:01.568428 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:01.589110 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:01.591650 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.591718 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:01.629249 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:02.423108 34543 executor.go:111] Tasks: 0 done / 96 total; 51 can run W1102 00:59:02.423584 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:02.424146 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:02.424197 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1102 00:59:02.424353 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:02.424380 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:02.425080 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:02.433056 34543 executor.go:111] Tasks: 51 done / 96 total; 19 can run I1102 00:59:02.438518 34543 executor.go:111] Tasks: 70 done / 96 total; 22 can run I1102 00:59:02.439194 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.439469 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.440612 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.442379 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.442506 34543 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1102 00:59:02.442895 34543 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1102 00:59:02.443073 34543 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1102 00:59:02.499123 34543 target.go:112] Terraform output is in /tmp/test1260429352/out --- PASS: TestCCM (0.94s) === RUN TestExternalDNS I1102 00:59:02.508726 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:02.508981 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:02.509043 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:02.509150 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:02.509208 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:02.509257 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:02.509287 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:02.509328 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:02.509360 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:02.538642 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:02.541727 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.541831 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.580515 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:02.686090 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:59:02.686756 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:02.686763 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:02.686810 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:02.686940 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:02.686973 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:02.687199 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:02.689671 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:02.695302 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:02.695616 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.695865 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.696861 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.698133 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.698254 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:02.698602 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:02.698799 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:02.752512 34543 target.go:112] Terraform output is in /tmp/test2048152147/out I1102 00:59:02.756383 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:02.756606 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:02.756663 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:02.756713 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:02.756767 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:02.756810 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:02.756838 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:02.756895 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:02.756927 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:02.781166 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:02.784229 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.784321 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.822246 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:02.925499 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:59:02.925889 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:02.926183 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:02.926226 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:02.926380 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:59:02.926437 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:02.926526 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:02.926703 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:02.926731 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:02.926861 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:02.930670 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:02.939490 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:02.940071 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.940371 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.941573 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.942628 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.942779 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:02.943181 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:02.943444 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:02.946917 34543 target.go:168] Cloudformation output is in /tmp/test4283885809/out --- PASS: TestExternalDNS (0.45s) === RUN TestExternalDNSIRSA I1102 00:59:02.956771 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:02.957036 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:02.957094 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:02.957143 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:02.957196 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:02.957278 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:02.957307 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:02.957344 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:02.957374 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:02.985672 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:02.988834 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:02.988929 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.026390 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:03.144129 34543 executor.go:111] Tasks: 0 done / 92 total; 46 can run W1102 00:59:03.144457 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:03.144651 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:03.144844 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:03.144885 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:03.144985 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:03.144992 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:03.146976 34543 executor.go:111] Tasks: 46 done / 92 total; 21 can run I1102 00:59:03.152093 34543 executor.go:111] Tasks: 67 done / 92 total; 21 can run I1102 00:59:03.152390 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.152692 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.153460 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.154949 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.155069 34543 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1102 00:59:03.155368 34543 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1102 00:59:03.155567 34543 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1102 00:59:03.211219 34543 target.go:112] Terraform output is in /tmp/test2981676405/out --- PASS: TestExternalDNSIRSA (0.26s) === RUN TestKarpenter I1102 00:59:03.215561 34543 featureflag.go:160] FeatureFlag "Karpenter"=true I1102 00:59:03.215606 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:03.215851 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:03.215907 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:03.215958 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:03.216008 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:03.216098 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:03.216128 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:03.216166 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:03.216198 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:03.250073 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:59:03.253972 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.254067 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.254130 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.293327 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:03.474606 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.474778 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.474901 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.592582 34543 executor.go:111] Tasks: 0 done / 108 total; 53 can run W1102 00:59:03.593005 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:03.593152 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:03.593197 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:03.593182 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:03.593291 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:03.595566 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:03.600279 34543 executor.go:111] Tasks: 53 done / 108 total; 26 can run I1102 00:59:03.605970 34543 executor.go:111] Tasks: 79 done / 108 total; 25 can run I1102 00:59:03.606385 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.606575 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.606826 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.607076 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.607626 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.610588 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.610831 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.611707 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.612021 34543 executor.go:111] Tasks: 104 done / 108 total; 2 can run I1102 00:59:03.612913 34543 executor.go:111] Tasks: 106 done / 108 total; 2 can run I1102 00:59:03.613705 34543 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1102 00:59:03.684052 34543 target.go:112] Terraform output is in /tmp/test987088922/out I1102 00:59:03.690346 34543 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.47s) === RUN TestSharedSubnet I1102 00:59:03.690480 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:03.690725 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:03.690788 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:03.690839 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:03.690894 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:03.690992 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:03.691023 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:03.691061 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:03.691091 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:03.713234 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:03.715666 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.715746 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.753480 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:03.854385 34543 executor.go:111] Tasks: 0 done / 80 total; 42 can run W1102 00:59:03.854979 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:03.854959 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:03.855329 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:03.855387 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:03.855416 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:03.855539 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:03.857825 34543 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1102 00:59:03.858179 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:03.867213 34543 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1102 00:59:03.867679 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.867910 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.868904 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.870149 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.870300 34543 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1102 00:59:03.870596 34543 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1102 00:59:03.870799 34543 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1102 00:59:03.915394 34543 target.go:112] Terraform output is in /tmp/test3593925363/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I1102 00:59:03.919667 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:03.919914 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:03.919973 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:03.920023 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:03.920076 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:03.920191 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:03.920245 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:03.920286 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:03.920319 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:03.947990 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:03.951094 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.951197 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:03.989628 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:04.086652 34543 executor.go:111] Tasks: 0 done / 84 total; 42 can run W1102 00:59:04.087197 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:04.087563 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:04.087587 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:04.087602 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:04.087612 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:04.087728 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:04.089650 34543 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1102 00:59:04.089967 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:04.096303 34543 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1102 00:59:04.096558 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.096852 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.098080 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.098911 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.099027 34543 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1102 00:59:04.099392 34543 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1102 00:59:04.099601 34543 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1102 00:59:04.147316 34543 target.go:112] Terraform output is in /tmp/test2038065415/out --- PASS: TestSharedVPC (0.23s) === RUN TestExistingIAM I1102 00:59:04.151385 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:04.151641 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:04.151704 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:04.151760 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:04.151817 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:04.151914 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:04.151944 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:04.151984 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:04.152013 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:04.183826 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:04.186536 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.186611 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.186658 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.186705 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.224670 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:04.330600 34543 executor.go:111] Tasks: 0 done / 101 total; 51 can run W1102 00:59:04.331315 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:04.331374 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:04.331350 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:04.331448 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:04.331609 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:04.331616 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:04.331661 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:04.331796 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:04.333706 34543 executor.go:111] Tasks: 51 done / 101 total; 15 can run I1102 00:59:04.338886 34543 executor.go:111] Tasks: 66 done / 101 total; 27 can run I1102 00:59:04.339184 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.341109 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.341497 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.341797 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.345784 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.346350 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.346493 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.346806 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:04.347037 34543 executor.go:111] Tasks: 93 done / 101 total; 4 can run I1102 00:59:04.347676 34543 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1102 00:59:04.347933 34543 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1102 00:59:04.421399 34543 target.go:112] Terraform output is in /tmp/test3835652382/out --- PASS: TestExistingIAM (0.27s) === RUN TestPhaseNetwork I1102 00:59:04.426072 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:04.426328 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:04.426394 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:04.426456 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:04.426500 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:04.426557 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:04.426598 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:04.426629 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:04.426654 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:04.459081 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:04.462837 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.462934 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.462998 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.501222 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:04.604200 34543 executor.go:111] Tasks: 0 done / 119 total; 46 can run I1102 00:59:04.604513 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:04.604654 34543 executor.go:111] Tasks: 46 done / 119 total; 28 can run I1102 00:59:04.605179 34543 executor.go:111] Tasks: 74 done / 119 total; 36 can run I1102 00:59:04.605592 34543 executor.go:111] Tasks: 110 done / 119 total; 5 can run I1102 00:59:04.605833 34543 executor.go:111] Tasks: 115 done / 119 total; 4 can run I1102 00:59:04.605994 34543 executor.go:111] Tasks: 119 done / 119 total; 0 can run I1102 00:59:04.615011 34543 target.go:112] Terraform output is in /tmp/test2134022044/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I1102 00:59:04.615833 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:04.616072 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:04.616132 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:04.616230 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:04.616283 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:04.616327 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:04.616361 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:04.616399 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:04.616427 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:04.641665 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:04.644837 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.644946 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.683452 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:04.782356 34543 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1102 00:59:04.782918 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:04.782924 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:04.783030 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:04.783065 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:04.783165 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:04.783469 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:04.785155 34543 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1102 00:59:04.794406 34543 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1102 00:59:04.794686 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.795082 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.796257 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.797133 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.797252 34543 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1102 00:59:04.797597 34543 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1102 00:59:04.797841 34543 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1102 00:59:04.852116 34543 target.go:112] Terraform output is in /tmp/test1479131455/out I1102 00:59:04.856189 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:04.856409 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:04.856465 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:04.856513 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:04.856567 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:04.856610 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:04.856639 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:04.856675 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:04.856700 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:04.882895 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:04.886253 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.886397 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:04.924773 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:05.026671 34543 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1102 00:59:05.027002 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:05.027252 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:05.027461 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:05.027495 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:05.027734 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:05.027652 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W1102 00:59:05.027310 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:05.027772 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:05.027879 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:59:05.030397 34543 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1102 00:59:05.035599 34543 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1102 00:59:05.036064 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.036271 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.037403 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.038349 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.038492 34543 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1102 00:59:05.038752 34543 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1102 00:59:05.038945 34543 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1102 00:59:05.041933 34543 target.go:168] Cloudformation output is in /tmp/test1462695216/out --- PASS: TestExternalLoadBalancer (0.44s) === RUN TestPhaseIAM integration_test.go:853: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:861: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I1102 00:59:05.056279 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:05.056554 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:05.056608 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:05.056657 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:05.056711 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:05.056794 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:05.056827 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:05.056885 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:05.056919 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:05.091055 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:05.094341 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094425 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094478 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094533 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094580 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094625 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.094675 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.133347 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:05.244521 34543 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1102 00:59:05.244796 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:05.245341 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:05.245383 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:05.245403 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:05.245427 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:05.245502 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:05.245599 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:05.245620 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:05.247812 34543 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1102 00:59:05.257586 34543 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1102 00:59:05.258073 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.258281 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.258491 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.258700 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.259726 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.260446 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.260857 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.260992 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.261139 34543 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1102 00:59:05.261589 34543 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1102 00:59:05.261868 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:59:05.344449 34543 target.go:112] Terraform output is in /tmp/test3379503882/out I1102 00:59:05.349511 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:05.349743 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:05.349806 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:05.349859 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:05.349914 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:05.349958 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:05.349985 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:05.350023 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:05.350052 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:05.382349 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:05.385475 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385561 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385615 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385668 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385717 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385761 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.385809 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.424360 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:05.539780 34543 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1102 00:59:05.540065 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:05.540083 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:05.540225 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:05.540242 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:05.540280 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:05.540436 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } W1102 00:59:05.540797 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:05.543322 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:05.543373 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:05.543495 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:59:05.544480 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:05.548240 34543 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1102 00:59:05.554399 34543 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1102 00:59:05.554757 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.554969 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.555279 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.555562 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.556312 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.557306 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.557747 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.558810 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.559259 34543 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1102 00:59:05.559953 34543 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1102 00:59:05.561384 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:59:05.567358 34543 target.go:168] Cloudformation output is in /tmp/test2610635761/out --- PASS: TestMixedInstancesASG (0.53s) === RUN TestMixedInstancesSpotASG I1102 00:59:05.584648 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:05.584886 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:05.584938 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:05.585029 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:05.585083 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:05.585126 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:05.585154 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:05.585194 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:05.585219 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:05.616766 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:05.619711 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.619788 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.619839 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.619886 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.619929 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.619969 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.620012 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.659681 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:05.774159 34543 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1102 00:59:05.774451 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:05.774572 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:05.774984 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:05.775106 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:05.775032 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1102 00:59:05.775268 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:05.775275 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:05.775332 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:05.777118 34543 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1102 00:59:05.788016 34543 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1102 00:59:05.788293 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.788612 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.788825 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.789166 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.790674 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.790919 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.791601 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.791691 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.791859 34543 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1102 00:59:05.792373 34543 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1102 00:59:05.792675 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:59:05.874026 34543 target.go:112] Terraform output is in /tmp/test2495876198/out I1102 00:59:05.879114 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:05.879341 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:05.879398 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:05.879446 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:05.879502 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:05.879544 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:05.879577 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:05.879614 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:05.879641 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:05.912010 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:05.915302 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915386 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915442 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915503 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915555 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915603 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.915650 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:05.954890 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:06.063356 34543 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1102 00:59:06.064347 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:06.064364 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.064369 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1102 00:59:06.064514 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.064434 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:06.064578 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:59:06.064597 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:06.064641 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:06.064792 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:06.064819 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:06.064971 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I1102 00:59:06.068181 34543 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1102 00:59:06.076423 34543 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1102 00:59:06.076796 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.077014 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.077255 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.077583 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.078092 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.080342 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.080703 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.081076 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:06.081228 34543 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1102 00:59:06.081576 34543 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1102 00:59:06.081900 34543 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1102 00:59:06.087524 34543 target.go:168] Cloudformation output is in /tmp/test2947441221/out --- PASS: TestMixedInstancesSpotASG (0.52s) === RUN TestAdditionalObjects I1102 00:59:06.107776 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:06.108041 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:06.108097 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:06.108147 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:06.108198 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:06.108286 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:06.108315 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:06.108353 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:06.108381 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:06.133977 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1102 00:59:06.136889 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.136978 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.175653 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:06.335870 34543 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1102 00:59:06.336124 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:06.336465 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:06.336491 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:06.337643 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:06.338305 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:06.339793 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:06.343448 34543 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1102 00:59:06.349008 34543 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1102 00:59:06.349547 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.349771 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.350911 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.353072 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1102 00:59:06.353203 34543 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1102 00:59:06.353553 34543 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1102 00:59:06.353732 34543 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1102 00:59:06.411034 34543 target.go:112] Terraform output is in /tmp/test1029898361/out --- PASS: TestAdditionalObjects (0.31s) === RUN TestContainerd I1102 00:59:06.415340 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:06.415584 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:06.415640 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:06.415743 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:06.415799 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:06.415843 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:06.415875 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:06.415913 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:06.415942 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:06.440430 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:06.443558 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.443657 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.482150 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:06.583122 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:59:06.583731 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:06.583904 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:06.583921 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:06.584029 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.584078 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:06.584035 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1102 00:59:06.584110 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:06.584224 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:59:06.584281 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.586206 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:06.595354 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:06.595605 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.595948 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.597171 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.598091 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.598232 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:06.598511 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:06.598746 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:06.602271 34543 target.go:168] Cloudformation output is in /tmp/test886049710/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I1102 00:59:06.612138 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:06.612391 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:06.612444 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:06.612533 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:06.612588 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:06.612632 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:06.612661 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:06.612701 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:06.612727 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:06.638262 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:06.641484 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.641571 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.680020 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:06.785350 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:59:06.785744 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:06.785968 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.785990 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:06.786019 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:06.786104 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:59:06.786177 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:06.786309 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:06.786327 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:06.788277 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1102 00:59:06.793481 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:06.799104 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:06.799536 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.799797 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.801250 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.802613 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.802772 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:06.803061 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:06.803268 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:06.806889 34543 target.go:168] Cloudformation output is in /tmp/test259438378/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I1102 00:59:06.816687 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:06.816937 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:06.816991 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:06.817104 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:06.817159 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:06.817202 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:06.817231 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:06.817268 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:06.817299 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:06.846936 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:06.850311 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.850412 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:06.889691 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:06.995302 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1102 00:59:06.995768 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.995902 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:06.995940 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:06.996062 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1102 00:59:06.995903 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:06.996375 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:06.996528 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:06.996552 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:06.996657 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I1102 00:59:06.999892 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:07.007901 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:07.008323 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:07.008604 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:07.009945 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:07.011183 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:07.011331 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:07.011622 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:07.011932 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:07.015250 34543 target.go:168] Cloudformation output is in /tmp/test4118006343/out --- PASS: TestDockerCustom (0.21s) === RUN TestAPIServerNodes I1102 00:59:07.026150 34543 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1102 00:59:07.026207 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:07.026439 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:07.026492 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:07.026582 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:07.026635 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:07.026678 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:07.026713 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:07.026752 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:07.026780 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:07.058007 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1102 00:59:07.061388 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:07.061483 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:07.061544 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.114147 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:08.277145 34543 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1102 00:59:08.277623 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:08.277635 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:08.278141 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:08.278187 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:08.278329 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:59:08.278334 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:08.278382 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:08.278468 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:08.278489 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:08.280336 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1102 00:59:08.284458 34543 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1102 00:59:08.291151 34543 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1102 00:59:08.291326 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.291582 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.291831 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.292732 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.293080 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.294283 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.294429 34543 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1102 00:59:08.294743 34543 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1102 00:59:08.294980 34543 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1102 00:59:08.298684 34543 target.go:168] Cloudformation output is in /tmp/test3252380945/out I1102 00:59:08.311431 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:08.311655 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:08.311712 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:08.311758 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:08.311809 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:08.311855 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:08.311884 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:08.311924 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:08.311953 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:08.342194 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1102 00:59:08.345591 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.345690 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.345752 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.384871 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:08.539716 34543 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1102 00:59:08.540159 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target W1102 00:59:08.540322 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:08.540300 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:08.540551 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:08.541954 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:08.542003 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:08.542198 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:08.547409 34543 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1102 00:59:08.552632 34543 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1102 00:59:08.553062 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.553270 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.553556 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.554763 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.555245 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.556213 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.556337 34543 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1102 00:59:08.556736 34543 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1102 00:59:08.557014 34543 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1102 00:59:08.622058 34543 target.go:112] Terraform output is in /tmp/test1436041944/out I1102 00:59:08.626589 34543 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.60s) === RUN TestNTHQueueProcessor I1102 00:59:08.626727 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:08.627001 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:08.627064 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:08.627171 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:08.627228 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:08.627275 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:08.627307 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:08.627345 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:08.627376 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:08.652320 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:08.655675 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:08.655772 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:09.967907 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:10.092719 34543 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1102 00:59:10.093056 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:10.093408 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:10.093636 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:10.093677 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:10.093816 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:10.093825 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:10.100678 34543 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1102 00:59:10.106471 34543 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1102 00:59:10.106705 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.107176 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.108368 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.109798 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.109921 34543 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1102 00:59:10.110236 34543 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1102 00:59:10.110664 34543 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1102 00:59:10.174676 34543 target.go:112] Terraform output is in /tmp/test3397854238/out I1102 00:59:10.179836 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:10.180081 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:10.180145 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:10.180218 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:10.180278 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:10.180326 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:10.180355 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:10.180396 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:10.180422 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:10.205200 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:10.208397 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.208495 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.248102 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1102 00:59:10.373408 34543 executor.go:111] Tasks: 0 done / 101 total; 45 can run W1102 00:59:10.373872 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1102 00:59:10.373889 34543 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1102 00:59:10.374383 34543 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1102 00:59:10.374425 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:10.375096 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:10.375671 34543 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1102 00:59:10.375697 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:10.375805 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:10.374560 34543 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1102 00:59:10.381080 34543 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1102 00:59:10.386638 34543 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1102 00:59:10.387407 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.387673 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.388832 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.389729 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.389870 34543 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1102 00:59:10.390299 34543 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1102 00:59:10.390758 34543 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1102 00:59:10.394632 34543 target.go:168] Cloudformation output is in /tmp/test2382264591/out --- PASS: TestNTHQueueProcessor (1.78s) === RUN TestCustomIRSA I1102 00:59:10.405476 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:10.405712 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:10.405766 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:10.405816 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:10.405867 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:10.405950 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:10.405978 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:10.406017 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:10.406044 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:10.440221 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:10.443668 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.443768 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:10.481215 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.035508 34543 executor.go:111] Tasks: 0 done / 100 total; 50 can run I1102 00:59:11.036102 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1102 00:59:11.036153 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:11.036249 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1102 00:59:11.036340 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:11.036364 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:11.036444 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:11.043916 34543 executor.go:111] Tasks: 50 done / 100 total; 25 can run I1102 00:59:11.049877 34543 executor.go:111] Tasks: 75 done / 100 total; 21 can run I1102 00:59:11.050347 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.050589 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.051451 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.053112 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.053235 34543 executor.go:111] Tasks: 96 done / 100 total; 2 can run I1102 00:59:11.053582 34543 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1102 00:59:11.053771 34543 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1102 00:59:11.115124 34543 target.go:112] Terraform output is in /tmp/test832673806/out --- PASS: TestCustomIRSA (0.72s) === RUN TestClusterNameDigit I1102 00:59:11.123927 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:11.124185 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:11.124250 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:11.124304 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:11.124361 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:11.124458 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:11.124489 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:11.124526 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:11.124557 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:11.151936 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:11.155255 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.155357 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.194184 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.301681 34543 executor.go:111] Tasks: 0 done / 95 total; 47 can run W1102 00:59:11.302149 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:11.302279 34543 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1102 00:59:11.302318 34543 warmpool.go:124] ASG warm pool is not supported by the terraform target I1102 00:59:11.302313 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:11.302445 34543 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1102 00:59:11.302778 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:11.304974 34543 executor.go:111] Tasks: 47 done / 95 total; 23 can run I1102 00:59:11.310093 34543 executor.go:111] Tasks: 70 done / 95 total; 21 can run I1102 00:59:11.310538 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.310797 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.311678 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.313026 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.313151 34543 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1102 00:59:11.313459 34543 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1102 00:59:11.313683 34543 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1102 00:59:11.370169 34543 target.go:112] Terraform output is in /tmp/test1118743408/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I1102 00:59:11.376740 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:11.377035 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:11.377100 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:11.377159 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:11.377215 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:11.377262 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:11.377291 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:11.377327 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:11.377353 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:11.377631 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I1102 00:59:11.401439 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:11.404428 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.404532 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:59:11.442872 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.542814 34543 iamrole.go:89] ListRoles: { } I1102 00:59:11.544506 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:59:11.544840 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I1102 00:59:11.544966 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1102 00:59:11.545300 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.545396 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:11.545521 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.545602 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I1102 00:59:11.545750 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I1102 00:59:11.545774 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.545747 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.546680 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:11.546842 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:11.547033 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:11.547393 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.547704 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.547892 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.548097 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:11.548176 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:11.548264 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:11.548354 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:11.548758 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:11.548998 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:11.549358 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:11.549565 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.549828 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:11.550965 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1102 00:59:11.551121 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1102 00:59:11.550113 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1102 00:59:11.551249 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1102 00:59:11.551390 34543 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1102 00:59:11.551426 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I1102 00:59:11.552272 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.552392 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.552486 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.552423 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1102 00:59:11.552562 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.552627 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.552619 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1102 00:59:11.552741 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1102 00:59:11.552854 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:11.552920 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:11.552960 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.553048 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:11.553187 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:11.553254 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:11.553333 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:11.553433 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:11.553500 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:11.553583 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:11.553642 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:11.553691 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:11.553857 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:11.553975 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:11.554022 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I1102 00:59:11.554426 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1102 00:59:11.556882 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:11.557140 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.557269 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.557455 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:11.557608 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.557841 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.557982 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.558111 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.558350 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.558488 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.558562 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.558613 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.558702 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.558850 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:11.558893 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.559047 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.559166 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:11.559254 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.559446 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:11.559502 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.559646 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.559810 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.560036 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I1102 00:59:11.560313 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1102 00:59:11.560550 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.560780 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1102 00:59:11.561788 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:11.562101 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:11.562240 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:11.562289 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.562329 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:11.562397 34543 dns.go:238] Pre-creating DNS records I1102 00:59:11.562431 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:11.568359 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:11.571300 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.571395 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1102 00:59:11.610052 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.712146 34543 iamrole.go:89] ListRoles: { } I1102 00:59:11.713722 34543 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1102 00:59:11.714027 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.714202 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:11.714267 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:11.714347 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.714313 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:11.714411 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:11.714501 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:11.716607 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.716980 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.717229 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.720663 34543 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1102 00:59:11.721554 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1102 00:59:11.721548 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.721839 34543 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1102 00:59:11.721701 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1102 00:59:11.721998 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:11.722096 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1102 00:59:11.722370 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:11.722475 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.722628 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:11.722785 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.723001 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.723238 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:11.726641 34543 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1102 00:59:11.726824 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.726965 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.727176 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.727362 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.727707 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.727886 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:11.728092 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:11.729787 34543 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1102 00:59:11.730076 34543 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1102 00:59:11.730257 34543 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1102 00:59:11.733276 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:11.733346 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:11.733386 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:11.733420 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:11.733456 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:11.733495 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:11.733553 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:11.733624 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:11.733664 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:11.733712 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:11.733753 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:11.733805 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:11.733852 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:11.733887 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:11.733919 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:11.733958 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:11.733992 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:11.734029 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:11.734067 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:11.734110 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:11.734145 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:11.734178 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:11.734211 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:11.734249 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:11.734287 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:11.734324 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:11.735631 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:11.735665 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:11.735709 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.735763 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1102 00:59:11.735819 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.735876 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.735926 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.735989 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.736039 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.736080 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1102 00:59:11.736128 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.736184 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1102 00:59:11.736228 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W1102 00:59:11.736275 34543 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W1102 00:59:11.736312 34543 api.go:53] PageSize not implemented I1102 00:59:11.736335 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:11.736370 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:11.736391 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:11.736421 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:11.736442 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:11.736499 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:11.736529 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:11.736581 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:11.736622 34543 iamrole.go:89] ListRoles: { } I1102 00:59:11.736689 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:11.736738 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:11.736776 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal.example.com ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I1102 00:59:11.737391 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:11.737483 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:11.737548 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:11.737479 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1102 00:59:11.737593 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:11.737618 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I1102 00:59:11.737684 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1102 00:59:11.737552 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I1102 00:59:11.737758 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I1102 00:59:11.737660 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1102 00:59:11.737914 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:11.737938 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:11.737981 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:11.738040 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1102 00:59:11.738155 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:11.738170 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:11.738204 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:11.738263 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I1102 00:59:11.738370 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I1102 00:59:11.738431 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I1102 00:59:11.738446 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1102 00:59:11.738414 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:11.738497 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I1102 00:59:11.738602 34543 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I1102 00:59:11.738622 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1102 00:59:11.738658 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I1102 00:59:11.738701 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I1102 00:59:11.738776 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:11.738860 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:11.738937 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:11.739286 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.36s) === RUN TestLifecycleMinimalOpenstack I1102 00:59:11.739671 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I1102 00:59:11.749480 34543 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:59:11.962275 34543 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1102 00:59:11.962317 34543 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1102 00:59:11.981869 34543 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1102 00:59:11.988920 34543 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1102 00:59:12.026613 34543 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1102 00:59:12.031576 34543 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1102 00:59:12.035274 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:12.035352 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:12.035463 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1102 00:59:12.040925 34543 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:59:12.227470 34543 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1102 00:59:12.227523 34543 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1102 00:59:12.247077 34543 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1102 00:59:12.256601 34543 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1102 00:59:12.268383 34543 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1102 00:59:12.271057 34543 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1102 00:59:12.274768 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:12.275151 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:12.275660 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1102 00:59:12.277242 34543 executor.go:111] Tasks: 96 done / 96 total; 0 can run Port:b390156d-c70e-414c-a1d4-0b1fb92b850e ok Port:49d27e51-2af4-4847-8c6d-88855c254c5b ok Subnet:fa8300dd-805f-4a70-bf3b-5c1f1559d774 ok ServerGroup:cba3388a-1f44-4aa5-822b-e99bed95df24 ok Instance:e7e5f0bc-9f5f-43ee-b6e0-579132dde461 ok SecurityGroup:de7810be-b243-46b6-bcbb-17100f7e4597 ok SecurityGroup:13a7fc9d-a431-4ca9-953c-b941d256e1c1 ok Port:20724131-1da3-4bb7-bbb3-438f4d58a3b6 ok ServerGroup:5dbdc78b-92f0-4bbc-b36e-faa35857a78f ok Instance:cfbb6a14-0ab6-43ec-ad48-2c6efc908f08 ok Network:c0b139ee-4a06-449c-90d5-418f20540288 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:f4745fde-2065-4eb6-b6f1-229c9e638223 ok SecurityGroup:a55b29dc-567d-42c0-8ab8-f36184abdc2a ok --- PASS: TestLifecycleMinimalOpenstack (1.65s) === RUN TestLifecycleMinimalGCE I1102 00:59:13.387801 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I1102 00:59:13.390446 34543 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1102 00:59:13.627764 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:59:13.627854 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:59:13.627972 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1102 00:59:13.628011 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1102 00:59:13.628040 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:59:13.628077 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:59:13.628154 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:59:13.629951 34543 executor.go:111] Tasks: 0 done / 64 total; 41 can run W1102 00:59:13.633091 34543 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I1102 00:59:13.633149 34543 executor.go:111] Tasks: 40 done / 64 total; 18 can run I1102 00:59:13.642725 34543 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1102 00:59:13.645829 34543 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1102 00:59:13.646060 34543 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1102 00:59:13.646157 34543 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1102 00:59:13.857810 34543 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1102 00:59:13.857892 34543 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1102 00:59:13.857983 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1102 00:59:13.858014 34543 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1102 00:59:13.858037 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:59:13.858066 34543 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1102 00:59:13.858128 34543 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1102 00:59:13.859652 34543 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1102 00:59:13.864069 34543 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1102 00:59:13.871871 34543 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1102 00:59:13.874758 34543 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1102 00:59:13.874939 34543 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1102 00:59:13.876576 34543 gce.go:98] Scanning zones: [us-test1-a] I1102 00:59:13.876993 34543 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I1102 00:59:13.877021 34543 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-master-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1667350753 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1667350753 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.49s) === RUN TestLifecycleFloatingIPOpenstack I1102 00:59:13.878654 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I1102 00:59:13.887808 34543 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:59:14.100360 34543 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1102 00:59:14.100413 34543 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1102 00:59:14.117644 34543 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1102 00:59:14.128609 34543 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1102 00:59:14.175697 34543 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1102 00:59:14.178892 34543 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1102 00:59:14.181545 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:14.181890 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:14.182034 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1102 00:59:14.186546 34543 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1102 00:59:14.383095 34543 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1102 00:59:14.383286 34543 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1102 00:59:14.404277 34543 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1102 00:59:14.407943 34543 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1102 00:59:14.420265 34543 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1102 00:59:14.422532 34543 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1102 00:59:14.425620 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:14.425867 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1102 00:59:14.426307 34543 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1102 00:59:14.427687 34543 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:1f9428c9-614c-439e-8a88-5411ee2fb0bc ok SecurityGroup:acf0cc64-55b5-4db9-9dbe-c193e7f61fe8 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:1e8e9500-b2f4-4fcc-887e-aa1671ea1ced ok Instance:eec3cf0d-3154-42eb-bf4b-671289d7da18 ok ServerGroup:693d0de3-0f08-4bd4-8590-a4f183de354b ok SecurityGroup:bbd6c266-268d-47bd-92f8-a9122f248392 ok Instance:4bcb68d3-6ffa-4923-a709-96cc8c71e6c5 ok SecurityGroup:eb1ca83d-696d-47eb-85dd-0e9cc30840e6 ok Port:793ab859-e829-4e68-a7b1-daacf13b8095 ok Subnet:1f9428c9-614c-439e-8a88-5411ee2fb0bc ok ServerGroup:32efc28a-f129-4625-b008-109854562144 ok Port:cf86b825-e03a-46e8-821f-8d897203e6a7 ok Router:3df27d03-4243-4b1c-9e4d-b8e0dc9c32d6 ok Network:84c254de-4049-4ff4-8caa-c8be1f5d3e13 ok Port:1d317052-b0a4-48b7-81f5-a4fa54d70f16 ok Port:76bad5c0-e1c4-475a-b53b-da9b4c464675 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.66s) === RUN TestLifecyclePrivateCalico I1102 00:59:15.541481 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:15.541753 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:15.541811 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:15.541861 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:15.541909 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:15.541952 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:15.541980 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:15.542017 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:15.542044 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:15.542115 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I1102 00:59:15.577276 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:59:15.581001 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:15.581112 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:15.581178 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:15.620811 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.077869 34543 iamrole.go:89] ListRoles: { } I1102 00:59:16.079874 34543 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1102 00:59:16.080292 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.080558 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:16.080716 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:16.080673 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I1102 00:59:16.080795 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.080795 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1102 00:59:16.080847 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.080908 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:16.080982 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I1102 00:59:16.080930 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.081068 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1102 00:59:16.081159 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I1102 00:59:16.081230 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1102 00:59:16.081184 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I1102 00:59:16.081458 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.082697 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.083094 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.083073 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.083502 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:16.083819 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:16.085739 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:16.085823 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:16.085859 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:16.085887 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:16.085932 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.087587 34543 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1102 00:59:16.087751 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.087854 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1102 00:59:16.087933 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1102 00:59:16.088016 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1102 00:59:16.088140 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1102 00:59:16.088142 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1102 00:59:16.088047 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1102 00:59:16.088277 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1102 00:59:16.088389 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1102 00:59:16.088500 34543 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1102 00:59:16.088406 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1102 00:59:16.088608 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1102 00:59:16.088756 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1102 00:59:16.088785 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.088995 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.089093 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.089192 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1102 00:59:16.089315 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] }] } I1102 00:59:16.089410 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.089509 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.089581 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.089555 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1102 00:59:16.089659 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1102 00:59:16.089662 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:16.089769 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1102 00:59:16.089902 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.090013 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.090102 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:16.090153 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:16.090261 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:16.090315 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:16.090459 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1102 00:59:16.090475 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.090594 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:16.090651 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:16.090699 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:16.090794 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:16.090850 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:16.090504 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1102 00:59:16.090891 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:16.090987 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.091087 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:16.091136 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:16.091178 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.091270 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:16.091319 34543 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I1102 00:59:16.091409 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:16.091464 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:16.091507 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:16.091559 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:16.091596 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1102 00:59:16.091679 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:16.091723 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:16.091768 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:16.094276 34543 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1102 00:59:16.094488 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } W1102 00:59:16.094645 34543 api.go:53] PageSize not implemented I1102 00:59:16.094658 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1102 00:59:16.094807 34543 api.go:53] PageSize not implemented I1102 00:59:16.094839 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:16.094890 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:16.094933 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.094872 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1102 00:59:16.095041 34543 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1102 00:59:16.095074 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1102 00:59:16.095115 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1102 00:59:16.095154 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1102 00:59:16.095213 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1102 00:59:16.095198 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.095337 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.095409 34543 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1102 00:59:16.095469 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.095500 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1102 00:59:16.095554 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1102 00:59:16.095558 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.095772 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1102 00:59:16.095883 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.096058 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:16.096285 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1102 00:59:16.096433 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.096569 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.096702 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.096730 34543 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.096814 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1102 00:59:16.097077 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.097218 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.097365 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1102 00:59:16.097507 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.097641 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" }] }] } I1102 00:59:16.097780 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:16.097883 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I1102 00:59:16.098026 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.098198 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:16.098319 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.098405 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:16.098533 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.098671 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1102 00:59:16.098810 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.098946 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:16.099089 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.099305 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1102 00:59:16.099544 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1102 00:59:16.099727 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.099960 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.100093 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I1102 00:59:16.100245 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1102 00:59:16.100394 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1102 00:59:16.100446 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" }] }] } I1102 00:59:16.100603 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1102 00:59:16.100648 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.100703 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.100754 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.100871 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1102 00:59:16.103603 34543 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1102 00:59:16.103717 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:16.103749 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.103850 34543 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:16.103885 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:16.104000 34543 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1102 00:59:16.104101 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:16.104147 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:16.104190 34543 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1102 00:59:16.104262 34543 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1102 00:59:16.104378 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.104508 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1102 00:59:16.104584 34543 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1102 00:59:16.104626 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.104675 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:16.104737 34543 dns.go:238] Pre-creating DNS records I1102 00:59:16.104763 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:16.111508 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1102 00:59:16.115190 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.115285 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.115351 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.154274 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.597762 34543 iamrole.go:89] ListRoles: { } I1102 00:59:16.600158 34543 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1102 00:59:16.600454 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.600587 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:16.600621 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:16.600663 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.600795 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.600911 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:16.600941 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.601043 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.601071 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.601154 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.602805 34543 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1102 00:59:16.603009 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1102 00:59:16.602941 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.603121 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1102 00:59:16.603348 34543 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1102 00:59:16.603265 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1102 00:59:16.603425 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1102 00:59:16.603561 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.603727 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.603875 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:16.603918 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1102 00:59:16.604071 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:16.604124 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:16.604218 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.604397 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1102 00:59:16.604598 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:16.604661 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.604758 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.604856 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:16.604909 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.605027 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1102 00:59:16.605173 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:16.605311 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:16.608248 34543 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1102 00:59:16.608561 34543 api.go:53] PageSize not implemented W1102 00:59:16.608601 34543 api.go:53] PageSize not implemented I1102 00:59:16.608635 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.608602 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.608740 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.608776 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.608785 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.608906 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1102 00:59:16.608953 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I1102 00:59:16.609053 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1102 00:59:16.609011 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.609123 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:16.609178 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1102 00:59:16.609227 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1102 00:59:16.609303 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.609417 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.609572 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.609792 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.609932 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.610027 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.610242 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.610325 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:16.612702 34543 executor.go:111] Tasks: 118 done / 127 total; 5 can run W1102 00:59:16.612896 34543 api.go:53] PageSize not implemented I1102 00:59:16.612878 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:16.612974 34543 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I1102 00:59:16.612921 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.613017 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.613117 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W1102 00:59:16.613150 34543 api.go:53] PageSize not implemented I1102 00:59:16.613178 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.613280 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1102 00:59:16.613389 34543 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1102 00:59:16.613568 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.613728 34543 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1102 00:59:16.617427 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:16.617491 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:16.617528 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:16.617563 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:16.617597 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:16.617636 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1102 00:59:16.617670 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:16.617703 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:16.617736 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:16.617768 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:16.617805 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:16.617841 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:16.617881 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:16.617917 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1102 00:59:16.617955 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:16.617991 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:16.618024 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:16.618058 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:16.618094 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:16.618126 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:16.618165 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:16.618202 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:16.618237 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:16.618272 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:16.618307 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:16.618345 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:16.618385 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:16.618419 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:16.618456 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:16.618488 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:16.618521 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:16.618558 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:16.618594 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:16.618629 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:16.620281 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:16.620308 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:16.620346 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620406 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1102 00:59:16.620477 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620536 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620583 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620632 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620673 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620716 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1102 00:59:16.620775 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620826 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:16.620854 34543 natgateway.go:117] DescribeNatGateways: { } I1102 00:59:16.620891 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1102 00:59:16.620930 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W1102 00:59:16.620983 34543 api.go:53] PageSize not implemented I1102 00:59:16.621001 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1102 00:59:16.621052 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:16.621080 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:16.621100 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:16.621124 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:16.621143 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:16.621191 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.621220 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:16.621264 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:16.621304 34543 iamrole.go:89] ListRoles: { } I1102 00:59:16.621348 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:16.621397 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:16.621440 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1102 00:59:16.621476 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:16.621512 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1102 00:59:16.622135 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:bastion.privatecalico.example.com ok I1102 00:59:16.622211 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:16.622189 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I1102 00:59:16.622276 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-group:nodes.privatecalico.example.com ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I1102 00:59:16.622306 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:16.622281 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } internet-gateway:igw-2 ok I1102 00:59:16.622366 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I1102 00:59:16.622369 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I1102 00:59:16.622406 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1102 00:59:16.622458 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I1102 00:59:16.622499 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1102 00:59:16.622544 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I1102 00:59:16.622362 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1102 00:59:16.622592 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:16.622612 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I1102 00:59:16.622463 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I1102 00:59:16.622668 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:16.622729 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1102 00:59:16.622822 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:16.622838 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:16.622876 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:16.622977 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1102 00:59:16.623119 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:16.623137 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1102 00:59:16.623188 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:16.623256 34543 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1102 00:59:16.623317 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:16.623420 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1102 00:59:16.623474 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:16.623453 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I1102 00:59:16.623564 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I1102 00:59:16.623523 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:16.623588 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1102 00:59:16.623653 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I1102 00:59:16.623695 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I1102 00:59:16.623634 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } I1102 00:59:16.623744 34543 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com W1102 00:59:16.623751 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:16.623759 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1102 00:59:16.623790 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I1102 00:59:16.623770 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1102 00:59:16.623843 34543 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I1102 00:59:16.623825 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } elastic-ip:eipalloc-1 ok I1102 00:59:16.623917 34543 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I1102 00:59:16.623914 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:16.623941 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1102 00:59:16.624073 34543 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I1102 00:59:16.623987 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W1102 00:59:16.624165 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:16.624180 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1102 00:59:16.624255 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:16.624312 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1102 00:59:16.624453 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:16.624545 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:16.624851 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.08s) === RUN TestLifecyclePrivateKopeio I1102 00:59:16.625205 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:16.625496 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:16.625551 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:16.625599 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:16.625648 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:16.625691 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:16.625721 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:16.625760 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:16.625792 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:16.625858 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I1102 00:59:16.659082 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:16.662454 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.662550 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.662613 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.701402 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.817819 34543 iamrole.go:89] ListRoles: { } I1102 00:59:16.824121 34543 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1102 00:59:16.824762 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I1102 00:59:16.824876 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1102 00:59:16.824960 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.825050 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I1102 00:59:16.825149 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.825253 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" }] } I1102 00:59:16.825296 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:16.825376 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.825465 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:16.825582 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1102 00:59:16.825693 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I1102 00:59:16.825592 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:16.825841 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:16.825926 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.826124 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.826328 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I1102 00:59:16.826383 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:16.826587 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:16.826652 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.826804 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.826944 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:16.827002 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:16.827036 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:16.827063 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:16.827106 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.827710 34543 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1102 00:59:16.828010 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1102 00:59:16.827956 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1102 00:59:16.828152 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1102 00:59:16.828236 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1102 00:59:16.828300 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1102 00:59:16.828407 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1102 00:59:16.828530 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1102 00:59:16.828459 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.828723 34543 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1102 00:59:16.828807 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.828933 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1102 00:59:16.828931 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I1102 00:59:16.828767 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I1102 00:59:16.829046 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:16.829054 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I1102 00:59:16.829131 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:16.829193 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.829453 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:16.829402 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:16.829540 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.829604 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1102 00:59:16.829697 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.829834 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.829872 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I1102 00:59:16.829982 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.830086 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:16.830139 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1102 00:59:16.830295 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1102 00:59:16.830408 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:16.830533 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1102 00:59:16.830668 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:16.830793 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] }] } I1102 00:59:16.830923 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1102 00:59:16.831047 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.831174 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I1102 00:59:16.831275 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:16.831358 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1102 00:59:16.831429 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:16.831477 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.831573 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:16.831626 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:16.831671 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1102 00:59:16.831758 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:16.831413 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1102 00:59:16.831826 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:16.831914 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:16.831964 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1102 00:59:16.832023 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:16.831931 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:16.832083 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:16.832126 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1102 00:59:16.832177 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1102 00:59:16.832144 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1102 00:59:16.832218 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.832309 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:16.832367 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:16.832421 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:16.832462 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:16.832514 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:16.832550 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:16.832598 34543 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I1102 00:59:16.832679 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:16.832752 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:16.832797 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:16.832861 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:16.832911 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1102 00:59:16.832956 34543 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I1102 00:59:16.833071 34543 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1102 00:59:16.833160 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.833308 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } W1102 00:59:16.833388 34543 api.go:53] PageSize not implemented W1102 00:59:16.833459 34543 api.go:53] PageSize not implemented I1102 00:59:16.833376 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:16.833552 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1102 00:59:16.833667 34543 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1102 00:59:16.833760 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1102 00:59:16.833816 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1102 00:59:16.833637 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.833901 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1102 00:59:16.834033 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1102 00:59:16.834129 34543 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1102 00:59:16.834215 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1102 00:59:16.833960 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.834259 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1102 00:59:16.834333 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1102 00:59:16.834336 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1102 00:59:16.834642 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1102 00:59:16.834873 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:16.835066 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1102 00:59:16.835254 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.835480 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.835649 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1102 00:59:16.835815 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.835987 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.836074 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.836228 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:16.836369 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1102 00:59:16.836498 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1102 00:59:16.836628 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.836744 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:16.836845 34543 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1102 00:59:16.836933 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.837157 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.837371 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.837492 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:16.837531 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" }] }] } I1102 00:59:16.837681 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:16.837724 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.837813 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" }] }] } I1102 00:59:16.837956 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I1102 00:59:16.837993 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I1102 00:59:16.838119 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.838195 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:16.838293 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.838426 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1102 00:59:16.838567 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1102 00:59:16.838605 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:16.838736 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1102 00:59:16.838830 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1102 00:59:16.838966 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1102 00:59:16.839014 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.839147 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I1102 00:59:16.839284 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:16.839444 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1102 00:59:16.839516 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1102 00:59:16.839773 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.839854 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1102 00:59:16.839917 34543 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1102 00:59:16.839997 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.840082 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I1102 00:59:16.840152 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.840253 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1102 00:59:16.840321 34543 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1102 00:59:16.840365 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I1102 00:59:16.840404 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1102 00:59:16.840527 34543 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1102 00:59:16.840675 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:16.840762 34543 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1102 00:59:16.840832 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:16.840909 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1102 00:59:16.842483 34543 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1102 00:59:16.842623 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:16.842711 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1102 00:59:16.842857 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:16.842935 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I1102 00:59:16.843003 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:16.843129 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I1102 00:59:16.843217 34543 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1102 00:59:16.843393 34543 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1102 00:59:16.843466 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:16.843521 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:16.843630 34543 dns.go:238] Pre-creating DNS records I1102 00:59:16.843678 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:16.850089 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:16.853648 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.853794 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.853894 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:16.891797 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.005023 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.007065 34543 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1102 00:59:17.007452 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:17.007442 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.007683 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.007930 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.008009 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.008161 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.008352 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:17.008385 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:17.008444 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.008495 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I1102 00:59:17.009900 34543 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1102 00:59:17.010039 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1102 00:59:17.010267 34543 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1102 00:59:17.010264 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.010397 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1102 00:59:17.010446 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1102 00:59:17.010381 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1102 00:59:17.012215 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:17.012301 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.012397 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1102 00:59:17.013197 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I1102 00:59:17.013460 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.013640 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] }] } I1102 00:59:17.013996 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] }] } I1102 00:59:17.014324 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.015574 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.016390 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1102 00:59:17.016538 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] }] } I1102 00:59:17.016646 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.016735 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.016894 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1102 00:59:17.017042 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.017080 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1102 00:59:17.017118 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:17.017152 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:17.017191 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:17.017225 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:17.017280 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:17.020108 34543 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1102 00:59:17.020285 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W1102 00:59:17.020416 34543 api.go:53] PageSize not implemented I1102 00:59:17.020439 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:17.020453 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.020504 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.020618 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1102 00:59:17.020668 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W1102 00:59:17.020761 34543 api.go:53] PageSize not implemented I1102 00:59:17.020781 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.020769 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:17.020858 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.020890 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1102 00:59:17.020986 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:17.021172 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.021312 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.021431 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.021757 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1102 00:59:17.021892 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1102 00:59:17.022012 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1102 00:59:17.022339 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:17.022482 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.022599 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:17.023592 34543 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1102 00:59:17.023682 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.023760 34543 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I1102 00:59:17.023690 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1102 00:59:17.023793 34543 api.go:53] PageSize not implemented I1102 00:59:17.023822 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1102 00:59:17.023836 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.023946 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W1102 00:59:17.024035 34543 api.go:53] PageSize not implemented I1102 00:59:17.024060 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.024241 34543 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1102 00:59:17.024383 34543 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1102 00:59:17.028035 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:17.028104 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:17.028144 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:17.028179 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:17.028221 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:17.028289 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:17.028351 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:17.028415 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:17.028487 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:17.028556 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:17.028624 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:17.028694 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:17.028765 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1102 00:59:17.028847 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:17.028911 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:17.028975 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:17.029048 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:17.029109 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:17.029180 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1102 00:59:17.029260 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:17.029332 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:17.029399 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:17.029442 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:17.029484 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:17.029524 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.029564 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:17.029600 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:17.029640 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1102 00:59:17.029680 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1102 00:59:17.029724 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:17.029760 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:17.029795 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:17.029835 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:17.029875 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:17.029912 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:17.031909 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:17.031971 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:17.032069 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032235 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1102 00:59:17.032425 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032569 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032669 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032738 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032787 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.032843 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1102 00:59:17.032937 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.033010 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:17.033048 34543 natgateway.go:117] DescribeNatGateways: { } I1102 00:59:17.033091 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1102 00:59:17.033137 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W1102 00:59:17.033192 34543 api.go:53] PageSize not implemented I1102 00:59:17.033211 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1102 00:59:17.033269 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:17.033298 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:17.033318 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:17.033343 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:17.033362 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:17.033414 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.033445 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.033497 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:17.033552 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.033602 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:17.033652 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:17.033702 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I1102 00:59:17.033749 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1102 00:59:17.033786 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok autoscaling-group:nodes.privatekopeio.example.com ok I1102 00:59:17.034452 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1102 00:59:17.034550 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1102 00:59:17.034521 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-config:lt-3 ok I1102 00:59:17.034564 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I1102 00:59:17.034604 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok I1102 00:59:17.034734 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I1102 00:59:17.034721 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } iam-instance-profile:masters.privatekopeio.example.com ok load-balancer:bastion-privatekopeio-exa-d8ef8e ok I1102 00:59:17.034807 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1102 00:59:17.034472 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok I1102 00:59:17.034923 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I1102 00:59:17.034656 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1102 00:59:17.035106 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.035150 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1102 00:59:17.035213 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:17.035292 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:17.035329 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:17.035395 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:17.035454 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.035509 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1102 00:59:17.035625 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.035640 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:17.035673 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:17.035735 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:17.035783 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1102 00:59:17.035862 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.035876 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1102 00:59:17.036001 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I1102 00:59:17.036057 34543 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I1102 00:59:17.036004 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:17.036097 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1102 00:59:17.036080 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I1102 00:59:17.036162 34543 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1102 00:59:17.036151 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.036182 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1102 00:59:17.036219 34543 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I1102 00:59:17.036199 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1102 00:59:17.036280 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.036256 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I1102 00:59:17.036292 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:17.036325 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1102 00:59:17.036300 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1102 00:59:17.036372 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:17.036427 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1102 00:59:17.036416 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:17.036497 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok W1102 00:59:17.036514 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.036566 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1102 00:59:17.036616 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I1102 00:59:17.036565 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1102 00:59:17.036675 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I1102 00:59:17.036776 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:17.036832 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I1102 00:59:17.036913 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1102 00:59:17.036990 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:17.037062 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:17.037346 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.41s) === RUN TestLifecycleIPv6 I1102 00:59:17.037655 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:17.037929 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:17.037975 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:17.038017 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:17.038068 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:17.038111 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:17.038140 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:17.038177 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:17.038201 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:17.038265 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I1102 00:59:17.064040 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:17.067516 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.067614 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.105532 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.276643 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.278545 34543 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1102 00:59:17.278993 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:17.279237 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I1102 00:59:17.279314 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.279374 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I1102 00:59:17.279555 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.279532 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.279558 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.279660 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:17.279656 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1102 00:59:17.279704 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:17.279750 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:17.279833 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.279863 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.280033 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:59:17.280243 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.280419 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.280544 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.280664 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.280794 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:17.280839 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:17.281115 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:17.281215 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:17.281435 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:17.281892 34543 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1102 00:59:17.282068 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.282155 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.282229 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:17.282227 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1102 00:59:17.282302 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1102 00:59:17.282322 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1102 00:59:17.282425 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1102 00:59:17.282523 34543 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1102 00:59:17.282418 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:17.282429 34543 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1102 00:59:17.282469 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.282680 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.282811 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:17.282861 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.282944 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I1102 00:59:17.283047 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:17.283047 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1102 00:59:17.283215 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.283295 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1102 00:59:17.283507 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I1102 00:59:17.283608 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.283706 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:17.283785 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:17.283821 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:17.283864 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:17.284026 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:17.284093 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:17.284139 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.284226 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:17.284273 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:17.284318 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.284354 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:17.284400 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.284437 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:17.284483 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1102 00:59:17.284559 34543 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I1102 00:59:17.284436 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1102 00:59:17.286701 34543 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1102 00:59:17.286927 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.287080 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.287257 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1102 00:59:17.287330 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.287636 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I1102 00:59:17.287922 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.288091 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1102 00:59:17.288196 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.288287 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.288361 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.288458 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.288621 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.288605 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I1102 00:59:17.288686 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.288809 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.288868 34543 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.288999 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.289158 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1102 00:59:17.289257 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.289400 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.289534 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.289662 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:59:17.289803 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:17.289894 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:59:17.290031 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.290195 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1102 00:59:17.290264 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:59:17.290402 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:17.290457 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:17.290498 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.290587 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.290725 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.290854 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1102 00:59:17.291000 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I1102 00:59:17.291138 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1102 00:59:17.291292 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.291350 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:17.291403 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.293683 34543 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1102 00:59:17.293907 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:17.294065 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:17.293920 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.294216 34543 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.294367 34543 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I1102 00:59:17.294463 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.294536 34543 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I1102 00:59:17.294634 34543 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I1102 00:59:17.294767 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:17.294864 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.294847 34543 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1102 00:59:17.294970 34543 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1102 00:59:17.295076 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:17.295149 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1102 00:59:17.295314 34543 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1102 00:59:17.295499 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.295498 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.295736 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.296616 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.296743 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1102 00:59:17.296783 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.296927 34543 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1102 00:59:17.296992 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.297029 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.297084 34543 dns.go:238] Pre-creating DNS records I1102 00:59:17.297111 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.304017 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:17.308514 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.308670 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.346860 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.518844 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.520798 34543 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1102 00:59:17.521015 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.521087 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.521161 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:17.521212 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.521306 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:17.521337 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:17.521384 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.521452 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I1102 00:59:17.521665 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.521822 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.525195 34543 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1102 00:59:17.525700 34543 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1102 00:59:17.525663 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.525805 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1102 00:59:17.525747 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1102 00:59:17.525965 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1102 00:59:17.525900 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.528313 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.528439 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:17.528563 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1102 00:59:17.528773 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.528834 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.529751 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1102 00:59:17.529936 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.530006 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1102 00:59:17.530347 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:17.533064 34543 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1102 00:59:17.533460 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.533660 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.533750 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:17.533791 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1102 00:59:17.533829 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1102 00:59:17.533949 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.534075 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.534266 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:17.534334 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.534491 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.534583 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.534780 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.534904 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.535159 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.538140 34543 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1102 00:59:17.538266 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.538396 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:17.538379 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:17.538471 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:17.538493 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1102 00:59:17.538543 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1102 00:59:17.538486 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.538580 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1102 00:59:17.538627 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1102 00:59:17.538663 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1102 00:59:17.538817 34543 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1102 00:59:17.538936 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:17.538934 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.539030 34543 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1102 00:59:17.539061 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:17.539087 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1102 00:59:17.539065 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:17.539157 34543 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1102 00:59:17.539184 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:17.539209 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1102 00:59:17.539310 34543 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1102 00:59:17.542527 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:17.542586 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:17.542623 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:17.542661 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:17.542695 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:17.542730 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1102 00:59:17.542765 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:17.542798 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:17.542830 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:17.542861 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:17.542898 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:17.542945 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:17.543011 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:17.543085 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:17.543146 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:17.543203 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:17.543254 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:17.543315 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:17.543370 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:17.543424 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:17.543484 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:17.543572 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:17.543638 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.543703 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:17.543769 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:17.543819 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:17.543877 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:17.543925 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:17.543966 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:17.544001 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:17.545545 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:17.545573 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:17.545612 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545665 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1102 00:59:17.545726 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545786 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545837 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545887 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545927 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.545973 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1102 00:59:17.546026 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.546072 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:17.546100 34543 natgateway.go:117] DescribeNatGateways: { } I1102 00:59:17.546141 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1102 00:59:17.546181 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W1102 00:59:17.546270 34543 api.go:53] PageSize not implemented I1102 00:59:17.546298 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:17.546352 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:17.546378 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1102 00:59:17.546447 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:17.546488 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:17.546514 34543 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I1102 00:59:17.546619 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.546656 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.546735 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:17.546788 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.546860 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:17.546967 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:17.547084 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1102 00:59:17.547150 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:17.547196 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-config:lt-2 ok I1102 00:59:17.547772 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1102 00:59:17.547835 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I1102 00:59:17.547787 34543 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I1102 00:59:17.547839 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:17.547877 34543 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } autoscaling-group:nodes.minimal-ipv6.example.com ok I1102 00:59:17.547934 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok autoscaling-config:lt-1 ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok volume:vol-2 ok I1102 00:59:17.548001 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.548072 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I1102 00:59:17.547956 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I1102 00:59:17.547988 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W1102 00:59:17.548302 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.548319 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1102 00:59:17.548363 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:17.548423 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:17.548496 34543 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1102 00:59:17.548601 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:17.548679 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:17.548726 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.548822 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:17.548862 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.548921 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W1102 00:59:17.549060 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.549076 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:17.549110 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:17.549205 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.549259 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1102 00:59:17.549274 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1102 00:59:17.549308 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I1102 00:59:17.549283 34543 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1102 00:59:17.549344 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I1102 00:59:17.549386 34543 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1102 00:59:17.549371 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I1102 00:59:17.549400 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } subnet:subnet-1 ok I1102 00:59:17.549425 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I1102 00:59:17.549485 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1102 00:59:17.549559 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:17.549639 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:17.549922 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I1102 00:59:17.550188 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:17.550471 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:17.550518 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:17.550556 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:17.550600 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:17.550642 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:17.550671 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:17.550709 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:17.550733 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:17.550792 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I1102 00:59:17.575142 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:17.578179 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.578272 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.615927 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.718078 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.719692 34543 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1102 00:59:17.720104 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I1102 00:59:17.720256 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1102 00:59:17.720247 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.720534 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.720566 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:17.720618 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I1102 00:59:17.720632 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.720766 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" }] } I1102 00:59:17.720836 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:17.720955 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:17.720990 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:17.721044 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:17.721229 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:17.721283 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.721463 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:17.721591 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:17.721639 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1102 00:59:17.722868 34543 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1102 00:59:17.722989 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:17.723132 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1102 00:59:17.723119 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.723248 34543 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1102 00:59:17.723262 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:17.723323 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1102 00:59:17.723415 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1102 00:59:17.723389 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I1102 00:59:17.723357 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I1102 00:59:17.723580 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1102 00:59:17.723690 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-12345678" } I1102 00:59:17.723864 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1102 00:59:17.724040 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1102 00:59:17.724090 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:17.724200 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:17.724255 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:17.724308 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1102 00:59:17.724418 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.724484 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.724533 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:17.724569 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:17.724647 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:17.724698 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.725532 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1102 00:59:17.727995 34543 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1102 00:59:17.728182 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.728368 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1102 00:59:17.728428 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.728514 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.728698 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:17.728754 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:17.728886 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.729004 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.729072 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1102 00:59:17.729260 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.729606 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1102 00:59:17.729814 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.730051 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.730233 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.730388 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:17.731472 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1102 00:59:17.731686 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.732031 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.732312 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.732553 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.732867 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1102 00:59:17.733145 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I1102 00:59:17.733446 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.733705 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.734164 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1102 00:59:17.736886 34543 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1102 00:59:17.737198 34543 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1102 00:59:17.737418 34543 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1102 00:59:17.737478 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.737520 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.737641 34543 dns.go:238] Pre-creating DNS records I1102 00:59:17.737674 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.743349 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:17.746795 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.746906 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.784917 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.885206 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.886922 34543 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1102 00:59:17.887315 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.887558 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:17.887733 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.889581 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:17.889805 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:17.889863 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:17.889932 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.890130 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:17.894556 34543 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1102 00:59:17.894688 34543 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1102 00:59:17.894771 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1102 00:59:17.895528 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1102 00:59:17.895494 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:17.895639 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.895843 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.895920 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.896081 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:17.896128 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1102 00:59:17.896299 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.900676 34543 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1102 00:59:17.900878 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.900993 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.901213 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.901421 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.901590 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:17.901989 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.902231 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.903791 34543 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1102 00:59:17.903981 34543 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1102 00:59:17.904090 34543 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1102 00:59:17.906967 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:17.907041 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:17.907097 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:17.907162 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:17.907213 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:17.907265 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:17.907315 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:17.907374 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:17.907424 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:17.907498 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:17.907566 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:17.907611 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:17.907669 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:17.907729 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:17.907789 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:17.907839 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:17.907901 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:17.907964 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:17.908015 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:17.908049 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:17.908081 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:17.908119 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:17.908156 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:17.909568 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:17.909599 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:17.909640 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909692 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1102 00:59:17.909749 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909808 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909853 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909898 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909941 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.909984 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1102 00:59:17.910031 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.910088 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1102 00:59:17.910126 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W1102 00:59:17.910175 34543 api.go:53] PageSize not implemented I1102 00:59:17.910222 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:17.910286 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:17.910327 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:17.910394 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:17.910437 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:17.910554 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:17.910612 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:17.910708 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:17.910799 34543 iamrole.go:89] ListRoles: { } I1102 00:59:17.910890 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:17.910975 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:17.911044 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-1 ok I1102 00:59:17.911493 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } autoscaling-group:nodes.sharedvpc.example.com ok I1102 00:59:17.911586 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I1102 00:59:17.911514 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:17.911645 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:17.911621 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I1102 00:59:17.911762 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1102 00:59:17.911865 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok iam-instance-profile:nodes.sharedvpc.example.com ok I1102 00:59:17.911707 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1102 00:59:17.911934 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.911960 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I1102 00:59:17.912011 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:17.912070 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:17.912124 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1102 00:59:17.912241 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:17.912256 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:17.912289 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:17.912382 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I1102 00:59:17.912440 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1102 00:59:17.912403 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I1102 00:59:17.912456 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } subnet:subnet-1 ok I1102 00:59:17.912496 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I1102 00:59:17.912529 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I1102 00:59:17.912572 34543 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I1102 00:59:17.912585 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1102 00:59:17.912613 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I1102 00:59:17.912660 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:17.912928 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.36s) === RUN TestLifecycleComplex I1102 00:59:17.913199 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:17.913474 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:17.913525 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:17.913596 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:17.913644 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:17.913683 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:17.913717 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:17.913755 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:17.913779 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:17.913844 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I1102 00:59:17.949374 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:17.953657 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.953761 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:17.993842 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.190603 34543 iamrole.go:89] ListRoles: { } I1102 00:59:18.192933 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:59:18.193241 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1102 00:59:18.193347 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] } I1102 00:59:18.193300 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.193503 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:18.193518 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.193742 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W1102 00:59:18.193866 34543 api.go:53] PageSize not implemented I1102 00:59:18.193898 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.193957 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I1102 00:59:18.194060 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I1102 00:59:18.194150 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1102 00:59:18.194181 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:18.194388 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:18.194662 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:18.194754 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:18.194891 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:18.194993 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.195178 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1102 00:59:18.195347 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:18.195475 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:18.195536 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:18.195577 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:18.195629 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:18.195813 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:18.195870 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.196348 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:59:18.196540 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:18.196598 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1102 00:59:18.196617 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:18.196599 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.196795 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:18.196755 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:18.196767 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I1102 00:59:18.196897 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1102 00:59:18.196983 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.196959 34543 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1102 00:59:18.197104 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:18.197015 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:18.197205 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:18.197154 34543 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1102 00:59:18.197268 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:18.197237 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1102 00:59:18.197314 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.197488 34543 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I1102 00:59:18.197473 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1102 00:59:18.197552 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.197678 34543 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I1102 00:59:18.197723 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:18.197955 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1102 00:59:18.198078 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:18.198197 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I1102 00:59:18.198279 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.198361 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:18.198465 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.198566 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.198635 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I1102 00:59:18.198723 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1102 00:59:18.198716 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1102 00:59:18.198818 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:18.198927 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:18.198901 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1102 00:59:18.198982 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1102 00:59:18.199955 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1102 00:59:18.199785 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:18.199829 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1102 00:59:18.200272 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:18.200359 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:18.200443 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1102 00:59:18.200600 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:18.200671 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:18.200737 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:18.200787 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:18.200965 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:18.201086 34543 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1102 00:59:18.201285 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I1102 00:59:18.201493 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:18.201608 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:18.201737 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:18.201852 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:18.201939 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1102 00:59:18.202050 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:18.205516 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:59:18.205729 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.205857 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.205882 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:18.206019 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:18.206012 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.206195 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I1102 00:59:18.206161 34543 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I1102 00:59:18.206328 34543 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1102 00:59:18.206434 34543 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1102 00:59:18.206354 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I1102 00:59:18.206512 34543 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.206548 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.206694 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:18.206748 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:18.206932 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1102 00:59:18.207188 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1102 00:59:18.207621 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1102 00:59:18.207903 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.207994 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.208128 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:18.208223 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.208349 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:18.208397 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1102 00:59:18.208556 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I1102 00:59:18.208682 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.208782 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1102 00:59:18.208974 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1102 00:59:18.209172 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I1102 00:59:18.209314 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I1102 00:59:18.209434 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1102 00:59:18.209611 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1102 00:59:18.209839 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1102 00:59:18.210098 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1102 00:59:18.210404 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1102 00:59:18.210488 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I1102 00:59:18.210685 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1102 00:59:18.210899 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1102 00:59:18.210995 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1102 00:59:18.211137 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1102 00:59:18.211178 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1102 00:59:18.211323 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I1102 00:59:18.211470 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I1102 00:59:18.211615 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1102 00:59:18.211704 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I1102 00:59:18.211792 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1102 00:59:18.211941 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I1102 00:59:18.211990 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1102 00:59:18.212085 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.212176 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1102 00:59:18.212327 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1102 00:59:18.212476 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.212522 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1102 00:59:18.212555 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1102 00:59:18.212596 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.215779 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:59:18.215933 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:18.216150 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:18.216286 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:59:18.216434 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:59:18.216514 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.216553 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:18.216610 34543 dns.go:238] Pre-creating DNS records I1102 00:59:18.216637 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:18.224537 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:18.228874 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.228978 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.268903 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.466681 34543 iamrole.go:89] ListRoles: { } I1102 00:59:18.469037 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:59:18.469363 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:18.469455 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.469605 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:18.469739 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } W1102 00:59:18.470011 34543 api.go:53] PageSize not implemented I1102 00:59:18.469904 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.470162 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:18.470230 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:18.470298 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.470341 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:18.470456 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:18.470552 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:18.470641 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.472414 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:59:18.472688 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:18.472644 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.472778 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.472795 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:18.472913 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:18.472922 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:18.472905 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I1102 00:59:18.473375 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:18.473447 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:18.473476 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:18.473495 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.473660 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:18.473825 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:18.473873 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:18.473916 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.474083 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.474177 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.474313 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:18.474371 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:18.474581 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:18.474644 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.474898 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:18.475201 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:18.475277 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.475413 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.478304 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:59:18.478613 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.478687 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:18.478792 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:18.478812 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.478799 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.478860 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.478898 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.478938 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:18.478971 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:18.479007 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.479118 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.479308 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.479425 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.479584 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.479740 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.479967 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.480087 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.480239 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.480399 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.483091 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:59:18.483202 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:18.483304 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:18.483333 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:18.483364 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.483465 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:59:18.483608 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I1102 00:59:18.489950 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.493640 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.505571 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:18.509741 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.509840 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.547526 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.743627 34543 iamrole.go:89] ListRoles: { } I1102 00:59:18.745544 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:59:18.745844 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W1102 00:59:18.746018 34543 api.go:53] PageSize not implemented I1102 00:59:18.745978 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.746155 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.746267 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:18.746374 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:18.746520 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.746666 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:18.746823 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.747650 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:18.747949 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:18.747985 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:18.748052 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.753881 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:59:18.754083 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.754181 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:18.754334 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:18.754181 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.754420 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:18.754540 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:18.754502 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.754285 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:18.754783 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:18.754692 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1102 00:59:18.754907 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:18.754960 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.755099 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:18.755144 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.755356 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:18.755524 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:18.755574 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.755706 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:18.755812 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.755977 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:18.756030 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:18.756287 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:18.756335 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:18.756448 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:18.756636 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:18.759563 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:59:18.759663 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.759722 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:18.759808 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:18.759830 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.759876 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.759915 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.759956 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:18.759991 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:18.760024 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.759762 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.760225 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I1102 00:59:18.760267 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:18.760304 34543 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I1102 00:59:18.760338 34543 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I1102 00:59:18.760268 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.760371 34543 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1102 00:59:18.760458 34543 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1102 00:59:18.760465 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.760524 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.760559 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.760893 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.761100 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.761184 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:18.761260 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:18.761332 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.761484 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.761661 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.764493 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:59:18.764615 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:18.764720 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:18.764744 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:18.764768 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:18.764855 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:59:18.765006 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1102 00:59:18.765060 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:18.765098 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:18.765156 34543 dns.go:238] Pre-creating DNS records I1102 00:59:18.773718 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:18.778033 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.778139 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:18.816283 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.012176 34543 iamrole.go:89] ListRoles: { } I1102 00:59:19.014346 34543 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1102 00:59:19.014619 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.014810 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:19.014951 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1102 00:59:19.015079 34543 api.go:53] PageSize not implemented I1102 00:59:19.014928 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.015157 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:19.015207 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:19.015287 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1102 00:59:19.015470 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.015613 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:19.015720 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:19.015814 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:19.015906 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.017456 34543 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1102 00:59:19.017576 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.017649 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:19.017729 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:19.017746 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.017730 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.017824 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:19.017881 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:19.017876 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.017960 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.018061 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:19.018234 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:19.018295 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.018393 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I1102 00:59:19.018555 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:19.018597 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:19.018739 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:19.018795 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.018882 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.019009 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.019144 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:19.019195 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.019343 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:19.019397 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:19.019603 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:19.019648 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.024386 34543 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1102 00:59:19.024707 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.024935 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:19.025040 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:19.025068 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.024924 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.025112 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.025231 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.025292 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.025331 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.025363 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.025335 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.025441 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.025562 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.025654 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.025769 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.025868 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.025959 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.026137 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.026188 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.033188 34543 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1102 00:59:19.033314 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:19.033422 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:19.033448 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:19.033471 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.033590 34543 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1102 00:59:19.033747 34543 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I1102 00:59:19.037874 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.041984 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.054120 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:19.058761 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.058860 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.097272 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.287396 34543 iamrole.go:89] ListRoles: { } I1102 00:59:19.289787 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1102 00:59:19.290288 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:19.290472 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.290613 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:19.290652 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W1102 00:59:19.290706 34543 api.go:53] PageSize not implemented I1102 00:59:19.290643 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:19.290715 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.290918 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.291178 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:19.291369 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.291614 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.291833 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:19.292005 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:19.292171 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:19.292872 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:19.293053 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:19.292999 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:19.293023 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:19.293151 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.293146 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.293275 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:19.293266 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:19.293292 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.293379 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.293480 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:19.293553 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.293659 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.293751 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.293891 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:19.293949 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.294036 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.294119 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:19.294288 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:19.294344 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:19.294488 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:19.294557 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:19.294645 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.294860 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I1102 00:59:19.295078 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.295305 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:19.298899 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:19.299088 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.299173 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:19.299256 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:19.299275 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.299321 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.299357 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.299401 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.299435 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.299474 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.299318 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.299800 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.299927 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.300109 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.300216 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.300399 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.300540 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.300624 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.300774 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.300933 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.303936 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.304017 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.304584 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:19.304736 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:19.304838 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:19.304862 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:19.304886 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.304973 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:19.305116 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:59:19.305168 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.305203 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:19.305269 34543 dns.go:238] Pre-creating DNS records I1102 00:59:19.313019 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:19.317342 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.317445 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.357328 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.548786 34543 iamrole.go:89] ListRoles: { } I1102 00:59:19.551170 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1102 00:59:19.551375 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:19.551482 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:19.551621 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:19.551825 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W1102 00:59:19.551999 34543 api.go:53] PageSize not implemented I1102 00:59:19.552050 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:19.551991 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1102 00:59:19.552239 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:19.552361 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1102 00:59:19.552486 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:19.552518 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:19.552563 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.552752 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1102 00:59:19.552891 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.553986 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:19.554565 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.554771 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:19.554719 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.554786 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:19.554843 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.554886 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.554971 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:19.554971 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:19.555060 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:19.555151 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:19.555186 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.555210 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:19.555267 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.555368 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:19.555536 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:19.555576 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.555759 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.555917 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:19.555959 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:19.556158 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:19.556213 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.556318 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.556431 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.556521 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.556711 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:19.561317 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:19.561436 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.561563 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.561621 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:19.561710 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:19.561732 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.561689 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.561779 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.561817 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.561859 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.561822 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.561895 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.561930 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.562116 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.562306 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.562477 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.562666 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.562813 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.562970 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.563309 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.566682 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:19.567326 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:19.567502 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:19.567547 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:19.567618 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.568600 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:19.568764 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I1102 00:59:19.575118 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.579145 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.579242 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.588976 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:19.593155 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.593251 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.593310 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.631368 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.819028 34543 iamrole.go:89] ListRoles: { } I1102 00:59:19.823118 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1102 00:59:19.823886 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:19.824044 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:19.824072 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:19.824203 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.824532 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } W1102 00:59:19.824755 34543 api.go:53] PageSize not implemented I1102 00:59:19.825184 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.825428 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:19.825502 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:19.825645 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.825831 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:19.825951 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:19.826068 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:19.826197 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.826906 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:19.827090 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:19.827084 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:19.827189 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.827241 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.827322 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:19.827360 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:19.827372 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:19.827365 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.827482 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.827573 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:19.827662 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:19.827831 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:19.827878 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.828028 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:19.828071 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I1102 00:59:19.828198 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:19.828366 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:19.828415 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.828613 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:19.828663 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.828796 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:19.828948 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:19.828991 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.829074 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.829145 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:19.832518 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:19.832815 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.832922 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.832942 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:19.833031 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:19.833050 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.833098 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.833138 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.833127 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.833179 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:19.833221 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:19.833255 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:19.833344 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.833481 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.833686 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.834021 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.834206 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.834358 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:19.834497 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:19.834662 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.836758 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:19.836879 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:19.836995 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:19.837034 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:19.837076 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:19.837244 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:19.837512 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:59:19.837596 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:19.837661 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:19.837761 34543 dns.go:238] Pre-creating DNS records I1102 00:59:19.846033 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:19.850383 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.850491 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.850558 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:19.888779 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.083328 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.085700 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1102 00:59:20.086050 34543 api.go:53] PageSize not implemented I1102 00:59:20.086124 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.086256 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:20.086304 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.086524 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:20.086589 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:20.086688 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.086980 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:20.087119 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.087255 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:20.087361 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:20.087456 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:20.087554 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:20.087650 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.089095 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:20.089366 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:20.089283 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:20.089531 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.089635 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:20.089660 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:20.089633 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:20.089767 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.089736 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.089845 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:20.089933 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.089946 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.090056 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.090148 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.090332 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:20.090381 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:20.090507 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.090635 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:20.090695 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.090930 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.091086 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.091306 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.091572 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:20.091649 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:20.091853 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.091913 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.094629 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:20.094867 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.095003 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.095083 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:20.095176 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:20.095198 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.095243 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.095280 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.095325 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.095362 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.095392 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.095393 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.095673 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.095850 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.096029 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.096127 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.096273 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.096411 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.096530 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.097456 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.103401 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:20.103634 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:20.103746 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:20.103776 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:20.103806 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.103902 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:20.104053 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I1102 00:59:20.107401 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.111253 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.111412 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.111591 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.122017 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:20.126216 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.126316 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.126378 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.126433 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.164997 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.356261 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.358641 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1102 00:59:20.358934 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:20.359064 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:20.359188 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:20.359216 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:20.359318 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1102 00:59:20.359538 34543 api.go:53] PageSize not implemented I1102 00:59:20.359457 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1102 00:59:20.359660 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.359826 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:20.359861 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:20.359925 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.359982 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1102 00:59:20.360166 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.360374 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:20.361989 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:20.362124 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:20.362155 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.362255 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.362284 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:20.362412 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:20.362298 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.362497 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:20.362517 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:20.362620 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.362729 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:20.362772 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:20.362836 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.362973 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.363067 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.363215 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.363370 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:20.363412 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:20.363575 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.363636 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.363726 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.363849 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.363988 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:20.364027 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.364111 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:20.364256 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.367415 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:20.367610 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.367771 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.367864 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:20.367943 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:20.367908 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.367962 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.368106 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.368168 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.368211 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.368249 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.368284 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.368348 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.368544 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.368667 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.368805 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.368904 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.369038 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.369265 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.369482 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.371351 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:20.371439 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:20.371526 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:20.371553 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:20.371582 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.371737 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:20.371942 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:59:20.371998 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.372034 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:20.372092 34543 dns.go:238] Pre-creating DNS records I1102 00:59:20.379946 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1102 00:59:20.384182 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.384283 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.384344 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.384400 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.422556 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.607994 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.610443 34543 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1102 00:59:20.610774 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1102 00:59:20.610957 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1102 00:59:20.611068 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W1102 00:59:20.611228 34543 api.go:53] PageSize not implemented I1102 00:59:20.611215 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1102 00:59:20.611279 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:20.611344 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1102 00:59:20.611458 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:20.611487 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:20.611517 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.611728 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1102 00:59:20.611874 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.612078 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.612263 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.613185 34543 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1102 00:59:20.613320 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1102 00:59:20.613321 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.613405 34543 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:20.613401 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.613482 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1102 00:59:20.613675 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:20.613732 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.613696 34543 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1102 00:59:20.613806 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.613847 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.613954 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:20.613994 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.614078 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.614206 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1102 00:59:20.614255 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1102 00:59:20.614446 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.614500 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.614599 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.614680 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.614808 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1102 00:59:20.614956 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:20.615004 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.615106 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.615194 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.615277 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:20.618750 34543 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1102 00:59:20.618898 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.619111 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:20.619176 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:20.619195 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.619230 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.619262 34543 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.619210 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.619299 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1102 00:59:20.619334 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1102 00:59:20.619364 34543 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1102 00:59:20.619335 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.619553 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.619662 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.619862 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:20.620104 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.620300 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.620392 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.620477 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.620554 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.623363 34543 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1102 00:59:20.623467 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:20.623586 34543 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1102 00:59:20.623617 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1102 00:59:20.623644 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.623758 34543 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1102 00:59:20.623955 34543 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1102 00:59:20.624321 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:20.624383 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:20.624417 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:20.624451 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:20.624482 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:20.624526 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:20.624565 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:20.624599 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:20.624633 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:20.624673 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:20.624714 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:20.624756 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:20.624789 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:20.624820 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:20.624860 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:20.624895 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:20.624934 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:20.624972 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:20.625009 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:20.625045 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:20.625086 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:20.625120 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:20.625157 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1102 00:59:20.625197 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:20.625230 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:20.625263 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:20.625302 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:20.625339 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:20.625375 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:20.628163 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:20.628196 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:20.628230 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628290 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1102 00:59:20.628358 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628418 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628465 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628519 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628562 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628608 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1102 00:59:20.628665 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628738 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1102 00:59:20.628781 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W1102 00:59:20.628836 34543 api.go:53] PageSize not implemented I1102 00:59:20.628857 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:20.628882 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:20.628898 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1102 00:59:20.628938 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:20.628962 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:20.628982 34543 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" ] } I1102 00:59:20.629054 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.629081 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:20.629122 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:20.629158 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.629202 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:20.629257 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:20.629306 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1102 00:59:20.629838 34543 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I1102 00:59:20.629888 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1102 00:59:20.629981 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I1102 00:59:20.629956 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok autoscaling-group:nodes.complex.example.com ok I1102 00:59:20.630048 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I1102 00:59:20.630034 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I1102 00:59:20.630099 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:20.630126 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I1102 00:59:20.629924 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I1102 00:59:20.630161 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-config:lt-1 ok I1102 00:59:20.630159 34543 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I1102 00:59:20.630242 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:20.630308 34543 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok subnet:subnet-3 ok I1102 00:59:20.630363 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:20.630433 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:20.630494 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 } ] } W1102 00:59:20.630855 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:20.630877 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I1102 00:59:20.630923 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.630979 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W1102 00:59:20.631185 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:20.631206 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:20.631241 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:20.631299 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:20.631382 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1102 00:59:20.631406 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I1102 00:59:20.631438 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:20.631494 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I1102 00:59:20.631562 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1102 00:59:20.631619 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I1102 00:59:20.631656 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I1102 00:59:20.631716 34543 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I1102 00:59:20.631730 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1102 00:59:20.631758 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I1102 00:59:20.631837 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:20.631928 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:20.632008 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:20.632315 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.72s) === RUN TestLifecycleExternalLB I1102 00:59:20.632631 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:20.632893 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:20.632943 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:20.632990 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:20.633042 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:20.633084 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:20.633117 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:20.633158 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:20.633187 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:20.633249 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I1102 00:59:20.663789 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:20.667255 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.667366 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.705270 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.801997 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.803474 34543 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1102 00:59:20.803806 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I1102 00:59:20.803818 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.803938 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1102 00:59:20.803924 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.803981 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:20.804072 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:20.804121 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1102 00:59:20.804037 34543 api.go:53] PageSize not implemented W1102 00:59:20.804560 34543 api.go:53] PageSize not implemented I1102 00:59:20.804570 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.805650 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.806108 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:20.806489 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:20.806722 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:20.807156 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:20.807199 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I1102 00:59:20.807310 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.807370 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1102 00:59:20.804152 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:20.804157 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:20.807688 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:20.807686 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.807786 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.807904 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.807932 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:20.808051 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:20.808156 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1102 00:59:20.808338 34543 api.go:53] PageSize not implemented I1102 00:59:20.804131 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.808431 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1102 00:59:20.808791 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:20.808976 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:20.810309 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:20.810419 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.811902 34543 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1102 00:59:20.812043 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1102 00:59:20.812032 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.812192 34543 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1102 00:59:20.812265 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1102 00:59:20.812210 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1102 00:59:20.812452 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1102 00:59:20.812553 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:20.812639 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.812844 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:20.812763 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1102 00:59:20.812895 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.813070 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1102 00:59:20.813179 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:20.813237 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1102 00:59:20.813465 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.813756 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.813864 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:20.813823 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I1102 00:59:20.813961 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:20.813988 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:20.814084 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:20.814156 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:20.814309 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:20.814372 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.814432 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:20.814484 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:20.814541 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.814633 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:20.814696 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" } ] } I1102 00:59:20.814517 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1102 00:59:20.818243 34543 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1102 00:59:20.818501 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.818737 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.818894 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.819034 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.819188 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:20.819327 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" }] }] } I1102 00:59:20.819493 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1102 00:59:20.819726 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.819844 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.819936 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.820007 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.820264 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.820396 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.820466 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.820681 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:20.820768 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.820940 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1102 00:59:20.821093 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.821237 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.821386 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.821519 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1102 00:59:20.821704 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.821881 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1102 00:59:20.822043 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:20.822100 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:20.822888 34543 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1102 00:59:20.823167 34543 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1102 00:59:20.823311 34543 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1102 00:59:20.823359 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.823405 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:20.823459 34543 dns.go:238] Pre-creating DNS records I1102 00:59:20.823488 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:20.829532 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:20.832652 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.832753 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.870647 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.971515 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.973220 34543 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1102 00:59:20.973484 34543 api.go:53] PageSize not implemented W1102 00:59:20.973737 34543 api.go:53] PageSize not implemented I1102 00:59:20.973694 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:20.973707 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.973849 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:20.973936 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:20.973964 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1102 00:59:20.974010 34543 api.go:53] PageSize not implemented I1102 00:59:20.973951 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.974083 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.974218 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.974226 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.974314 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1102 00:59:20.974361 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:20.974393 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:20.974442 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.974431 34543 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:20.974480 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:20.974521 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:20.974545 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:20.974686 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.978124 34543 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1102 00:59:20.978329 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1102 00:59:20.978491 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.978851 34543 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1102 00:59:20.978889 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.979143 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:20.979367 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1102 00:59:20.980237 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.980321 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1102 00:59:20.980483 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:20.980541 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.980674 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.980818 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:20.985435 34543 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1102 00:59:20.985656 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.985799 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.985978 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.986422 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.986823 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.987030 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.987173 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:20.989437 34543 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1102 00:59:20.989859 34543 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1102 00:59:20.990074 34543 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1102 00:59:20.993734 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:20.993797 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:20.993832 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:20.993867 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:20.993902 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:20.993938 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:20.993971 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:20.994005 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:20.994040 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:20.994075 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:20.994110 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:20.994150 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:20.994195 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:20.994230 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:20.994262 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:20.994297 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:20.994330 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:20.994365 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:20.994403 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:20.994443 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:20.994475 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:20.994507 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:20.994539 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:20.994575 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:20.994611 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:20.994645 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:20.996006 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:20.996034 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:20.996077 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996129 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1102 00:59:20.996183 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996239 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996285 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996342 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996384 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996424 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1102 00:59:20.996471 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996522 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1102 00:59:20.996560 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W1102 00:59:20.996612 34543 api.go:53] PageSize not implemented I1102 00:59:20.996633 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:20.996658 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:20.996676 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:20.996698 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:20.996714 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:20.996763 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:20.996793 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:20.996841 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:20.996899 34543 iamrole.go:89] ListRoles: { } I1102 00:59:20.996943 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:20.996985 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:20.997018 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:20.997456 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.externallb.example.com ok I1102 00:59:20.997508 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I1102 00:59:20.997604 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I1102 00:59:20.997526 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1102 00:59:20.997654 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:20.997672 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } I1102 00:59:20.997653 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } security-group:sg-2 ok autoscaling-config:lt-1 ok I1102 00:59:20.997704 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I1102 00:59:20.997723 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:20.997788 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:20.997644 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I1102 00:59:20.997842 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1102 00:59:20.998041 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:20.998063 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:20.998096 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1102 00:59:20.998184 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:20.998287 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:20.998341 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:20.998447 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I1102 00:59:20.998511 34543 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I1102 00:59:20.998529 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1102 00:59:20.998499 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:20.998566 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I1102 00:59:20.998636 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I1102 00:59:20.998694 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I1102 00:59:20.998711 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1102 00:59:20.998744 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I1102 00:59:20.998809 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:20.998889 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:20.998958 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:20.999283 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I1102 00:59:20.999553 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:20.999804 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:20.999850 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:20.999892 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:20.999938 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:20.999979 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:21.000010 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:21.000049 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:21.000076 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:21.000138 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I1102 00:59:21.024665 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.027764 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.027858 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.065708 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.167095 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.168537 34543 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1102 00:59:21.168795 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.168832 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.168895 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1102 00:59:21.169150 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:21.169068 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.169138 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I1102 00:59:21.169311 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:21.169300 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I1102 00:59:21.169417 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:21.169603 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:21.169668 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:21.169761 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.170044 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.170137 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.170391 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.170630 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:21.170704 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.171511 34543 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1102 00:59:21.171631 34543 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1102 00:59:21.171638 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1102 00:59:21.171822 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.171867 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.171971 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:21.172120 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:21.172196 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.172139 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1102 00:59:21.172245 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1102 00:59:21.172296 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.172436 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1102 00:59:21.172494 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1102 00:59:21.172750 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.172890 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:21.172955 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.173010 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:21.173064 34543 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I1102 00:59:21.174180 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1102 00:59:21.176907 34543 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1102 00:59:21.177125 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:21.177317 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.177446 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.177514 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.177827 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.178071 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.178222 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I1102 00:59:21.178424 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.178669 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I1102 00:59:21.178815 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.178950 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.179100 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:21.179603 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1102 00:59:21.179737 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.179869 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.179995 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1102 00:59:21.180135 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.180280 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.181407 34543 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1102 00:59:21.181689 34543 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1102 00:59:21.181850 34543 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1102 00:59:21.181903 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.181944 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.182001 34543 dns.go:238] Pre-creating DNS records I1102 00:59:21.182032 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.191789 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.194832 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.194928 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.232883 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.331140 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.332673 34543 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1102 00:59:21.333058 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.333184 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.333251 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:21.333364 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.333725 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.333903 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:21.333986 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:21.334017 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:21.337703 34543 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1102 00:59:21.337995 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1102 00:59:21.338268 34543 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1102 00:59:21.338403 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1102 00:59:21.338353 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1102 00:59:21.339384 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.339489 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:21.339617 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1102 00:59:21.339750 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.345563 34543 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1102 00:59:21.345834 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.346070 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.346328 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.346598 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:21.348590 34543 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1102 00:59:21.348822 34543 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1102 00:59:21.349042 34543 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1102 00:59:21.351942 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:21.352010 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:21.352047 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:21.352079 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:21.352114 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:21.352148 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:21.352182 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:21.352214 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:21.352251 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:21.352297 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:21.352335 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:21.352368 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:21.352401 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:21.352433 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:21.352473 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:21.352508 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:21.352545 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:21.352578 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:21.352611 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:21.352649 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:21.352685 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:21.354140 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:21.354176 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:21.354218 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354270 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1102 00:59:21.354326 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354385 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354426 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354470 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354513 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354549 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1102 00:59:21.354588 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354634 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1102 00:59:21.354671 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W1102 00:59:21.354723 34543 api.go:53] PageSize not implemented I1102 00:59:21.354746 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:21.354774 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:21.354795 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:21.354822 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:21.354843 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1102 00:59:21.354900 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.354931 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.354983 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:21.355022 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.355073 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:21.355113 34543 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.sharedsubnet.example.com ok autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I1102 00:59:21.355525 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.355533 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.355592 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I1102 00:59:21.355717 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1102 00:59:21.355628 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1102 00:59:21.355852 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.355632 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I1102 00:59:21.355873 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:21.355817 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I1102 00:59:21.355905 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1102 00:59:21.355977 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.356042 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1102 00:59:21.356172 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.356193 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:21.356228 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:21.356290 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:21.356373 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I1102 00:59:21.356427 34543 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1102 00:59:21.356444 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1102 00:59:21.356482 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I1102 00:59:21.356519 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.356562 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1102 00:59:21.356575 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1102 00:59:21.356605 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I1102 00:59:21.356858 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedSubnet I1102 00:59:21.357146 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:21.357419 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:21.357471 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:21.357515 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:21.357561 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:21.357605 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:21.357637 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:21.357674 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:21.357703 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:21.357768 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I1102 00:59:21.386162 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.389515 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.389612 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.389678 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.427973 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.530719 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.532988 34543 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1102 00:59:21.533322 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:21.533446 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:21.533477 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:21.533455 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I1102 00:59:21.533519 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.533596 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.533592 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1102 00:59:21.533572 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:21.533727 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.533781 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.533881 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1102 00:59:21.533891 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.534138 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.534217 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.534359 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1102 00:59:21.534343 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.534493 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:21.534544 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.534643 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:21.536754 34543 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1102 00:59:21.536948 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1102 00:59:21.537054 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.537212 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1102 00:59:21.537159 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1102 00:59:21.537477 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.537606 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:21.537572 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.537661 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.537711 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1102 00:59:21.537924 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1102 00:59:21.537953 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1102 00:59:21.538113 34543 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1102 00:59:21.538020 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:21.538176 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.538214 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1102 00:59:21.538272 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1102 00:59:21.538490 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1102 00:59:21.538710 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.538845 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1102 00:59:21.538987 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.539110 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1102 00:59:21.539203 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:21.539262 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.539373 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:21.539428 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:21.539564 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.539649 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:21.539744 34543 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I1102 00:59:21.539910 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:21.539551 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.540011 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:21.540078 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.540232 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:21.540308 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:21.540373 34543 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" } ] } I1102 00:59:21.540507 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:21.546738 34543 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1102 00:59:21.547027 34543 api.go:53] PageSize not implemented I1102 00:59:21.546986 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.547196 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1102 00:59:21.547211 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.547309 34543 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I1102 00:59:21.547459 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1102 00:59:21.547421 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W1102 00:59:21.547547 34543 api.go:53] PageSize not implemented I1102 00:59:21.547594 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1102 00:59:21.547567 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.547751 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.547805 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1102 00:59:21.547938 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1102 00:59:21.547890 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.548089 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.548153 34543 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1102 00:59:21.548256 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.548217 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:21.548447 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.548304 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1102 00:59:21.548611 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:21.548672 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1102 00:59:21.548778 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.548989 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1102 00:59:21.549129 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.549306 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:21.549463 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.549649 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:21.549786 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.549963 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I1102 00:59:21.550109 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.550258 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.550379 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I1102 00:59:21.550583 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.550732 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.550869 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.551006 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1102 00:59:21.551152 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.551258 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.551339 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.551558 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1102 00:59:21.551722 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" }] }] } I1102 00:59:21.551883 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:21.552925 34543 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1102 00:59:21.553169 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:21.553367 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:21.553554 34543 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1102 00:59:21.553782 34543 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1102 00:59:21.553849 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.553887 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.553943 34543 dns.go:238] Pre-creating DNS records I1102 00:59:21.553970 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.561751 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.565468 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.565561 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.565630 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.604782 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.706555 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.708859 34543 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1102 00:59:21.709308 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:21.709403 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:21.709556 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:21.709608 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:21.709676 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.709840 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.709973 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.710215 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.711617 34543 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1102 00:59:21.711820 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1102 00:59:21.711918 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1102 00:59:21.712030 34543 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1102 00:59:21.712128 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1102 00:59:21.712086 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1102 00:59:21.712280 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.712312 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1102 00:59:21.712344 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1102 00:59:21.712493 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:21.712552 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.712680 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1102 00:59:21.712999 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:21.713063 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1102 00:59:21.713288 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1102 00:59:21.713408 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:21.713624 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:21.716686 34543 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1102 00:59:21.717071 34543 api.go:53] PageSize not implemented I1102 00:59:21.717081 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.717116 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } W1102 00:59:21.717250 34543 api.go:53] PageSize not implemented I1102 00:59:21.717283 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.717289 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.717397 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.717437 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1102 00:59:21.717511 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1102 00:59:21.717592 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1102 00:59:21.717590 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.717759 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.718014 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.718135 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.719486 34543 executor.go:111] Tasks: 100 done / 107 total; 4 can run W1102 00:59:21.719614 34543 api.go:53] PageSize not implemented I1102 00:59:21.719578 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:21.719664 34543 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I1102 00:59:21.719650 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.719833 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W1102 00:59:21.719894 34543 api.go:53] PageSize not implemented I1102 00:59:21.719924 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.720140 34543 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1102 00:59:21.720312 34543 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1102 00:59:21.723881 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:21.723948 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:21.723984 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:21.724020 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:21.724054 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:21.724088 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:21.724119 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:21.724153 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:21.724193 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:21.724234 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:21.724273 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1102 00:59:21.724317 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:21.724352 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:21.724385 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:21.724418 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:21.724457 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:21.724494 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:21.724530 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:21.724568 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:21.724605 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:21.724643 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:21.724680 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:21.724715 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:21.724753 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:21.724792 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:21.726394 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:21.726428 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:21.726471 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726540 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1102 00:59:21.726621 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726681 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726728 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726772 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726817 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726854 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1102 00:59:21.726929 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.726977 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1102 00:59:21.727012 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W1102 00:59:21.727061 34543 api.go:53] PageSize not implemented I1102 00:59:21.727079 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1102 00:59:21.727132 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:21.727161 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:21.727180 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:21.727204 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:21.727224 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:21.727276 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.727305 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.727352 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:21.727392 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.727439 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:21.727485 34543 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-2 ok I1102 00:59:21.728024 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.728095 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } autoscaling-group:bastion.private-shared-subnet.example.com ok iam-instance-profile:bastions.private-shared-subnet.example.com ok I1102 00:59:21.728103 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } autoscaling-config:lt-3 ok load-balancer:api-private-shared-subnet-n2f8ak ok I1102 00:59:21.728131 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } autoscaling-config:lt-1 ok I1102 00:59:21.728181 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I1102 00:59:21.728200 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:21.728216 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.728264 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:21.728298 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I1102 00:59:21.728289 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I1102 00:59:21.728318 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1102 00:59:21.728390 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.private-shared-subnet.example.com ok load-balancer:bastion-private-shared-su-5ol32q ok I1102 00:59:21.728456 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I1102 00:59:21.728391 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W1102 00:59:21.728587 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok I1102 00:59:21.728611 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:21.728643 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:21.728724 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1102 00:59:21.728817 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.728832 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1102 00:59:21.728870 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.728959 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1102 00:59:21.729173 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.729191 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:21.729264 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:21.729315 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.729408 34543 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1102 00:59:21.729431 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1102 00:59:21.729322 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I1102 00:59:21.729470 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok W1102 00:59:21.729487 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.729503 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1102 00:59:21.729504 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.729547 34543 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1102 00:59:21.729560 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1102 00:59:21.729589 34543 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I1102 00:59:21.729550 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:21.729617 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I1102 00:59:21.729680 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1102 00:59:21.729694 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1102 00:59:21.729726 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I1102 00:59:21.729636 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1102 00:59:21.729764 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:21.729783 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1102 00:59:21.730055 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedIP I1102 00:59:21.730345 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:21.730631 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:21.730682 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:21.730744 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:21.730796 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:21.730838 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:21.730865 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:21.730905 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:21.730930 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:21.730994 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I1102 00:59:21.758821 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.762194 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.762289 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.762354 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.800767 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.901217 34543 iamrole.go:89] ListRoles: { } I1102 00:59:21.903378 34543 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1102 00:59:21.903752 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:21.903747 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1102 00:59:21.903867 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.903911 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1102 00:59:21.904149 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I1102 00:59:21.904114 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.904275 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1102 00:59:21.904406 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I1102 00:59:21.904405 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:21.904519 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1102 00:59:21.904568 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:21.904628 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:21.904671 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.904857 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.904946 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:21.905120 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:21.905239 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:21.905413 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:21.905471 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.906759 34543 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1102 00:59:21.907015 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1102 00:59:21.907071 34543 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1102 00:59:21.907111 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1102 00:59:21.906957 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1102 00:59:21.907146 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1102 00:59:21.907155 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:21.907321 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1102 00:59:21.907298 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.907548 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:21.907638 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:21.907709 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.907818 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1102 00:59:21.907896 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:21.908003 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1102 00:59:21.908153 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1102 00:59:21.908073 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1102 00:59:21.908107 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I1102 00:59:21.908325 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:21.908376 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.908517 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:21.908575 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:21.908639 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1102 00:59:21.908744 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.909002 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I1102 00:59:21.909004 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1102 00:59:21.909246 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1102 00:59:21.909485 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.909603 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1102 00:59:21.909472 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1102 00:59:21.909776 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.909963 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:21.910096 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:21.910155 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:21.910193 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.910325 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:21.910277 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I1102 00:59:21.910380 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:21.910454 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:21.910520 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:21.910565 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1102 00:59:21.910665 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:21.910711 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:21.910764 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1102 00:59:21.910958 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I1102 00:59:21.911023 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:21.911139 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:21.911210 34543 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I1102 00:59:21.912056 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:21.917039 34543 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1102 00:59:21.917188 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1102 00:59:21.917484 34543 api.go:53] PageSize not implemented I1102 00:59:21.917408 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } W1102 00:59:21.917606 34543 api.go:53] PageSize not implemented I1102 00:59:21.917693 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:21.917768 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1102 00:59:21.917831 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:21.917886 34543 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1102 00:59:21.917966 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1102 00:59:21.917976 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1102 00:59:21.918000 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1102 00:59:21.918060 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1102 00:59:21.918147 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1102 00:59:21.918222 34543 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1102 00:59:21.918114 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I1102 00:59:21.918280 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1102 00:59:21.918310 34543 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1102 00:59:21.918358 34543 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1102 00:59:21.918333 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.918652 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I1102 00:59:21.918806 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:21.918932 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1102 00:59:21.919083 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:21.919134 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.919234 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I1102 00:59:21.919376 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.919523 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.919610 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1102 00:59:21.919692 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:21.919766 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:21.919852 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.920006 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1102 00:59:21.920226 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.920386 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.920478 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.920642 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.920800 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:21.920837 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1102 00:59:21.920962 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.921099 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.921146 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.921189 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1102 00:59:21.921327 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:21.921365 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1102 00:59:21.921414 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1102 00:59:21.921457 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I1102 00:59:21.921620 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1102 00:59:21.921749 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.921878 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.922036 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.922185 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I1102 00:59:21.922321 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.922460 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.922506 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1102 00:59:21.922537 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:21.922579 34543 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I1102 00:59:21.923757 34543 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1102 00:59:21.923851 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:21.923976 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:21.924009 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:21.924096 34543 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:21.924267 34543 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1102 00:59:21.924397 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:21.924474 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:21.924546 34543 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1102 00:59:21.924653 34543 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1102 00:59:21.924854 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:21.925018 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1102 00:59:21.925099 34543 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1102 00:59:21.925146 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:21.925180 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.925236 34543 dns.go:238] Pre-creating DNS records I1102 00:59:21.925263 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:21.932439 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1102 00:59:21.936524 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.936671 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.936788 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:21.975361 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.075471 34543 iamrole.go:89] ListRoles: { } I1102 00:59:22.077144 34543 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1102 00:59:22.077355 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1102 00:59:22.077457 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1102 00:59:22.077479 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1102 00:59:22.078596 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:22.079710 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.080356 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.082496 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.082615 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.083844 34543 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1102 00:59:22.084020 34543 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1102 00:59:22.084112 34543 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1102 00:59:22.084101 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.084284 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1102 00:59:22.084299 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1102 00:59:22.084639 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:22.084723 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1102 00:59:22.084968 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:22.085046 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.085237 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1102 00:59:22.085414 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:22.085489 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1102 00:59:22.085735 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1102 00:59:22.085800 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1102 00:59:22.085961 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1102 00:59:22.086110 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1102 00:59:22.086175 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.086318 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.086501 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:22.086606 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:22.089652 34543 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1102 00:59:22.089990 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W1102 00:59:22.090071 34543 api.go:53] PageSize not implemented I1102 00:59:22.090120 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:22.090168 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1102 00:59:22.090335 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.090393 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W1102 00:59:22.090493 34543 api.go:53] PageSize not implemented I1102 00:59:22.090547 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1102 00:59:22.090568 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:22.090662 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1102 00:59:22.090709 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1102 00:59:22.090739 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:22.090755 34543 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1102 00:59:22.090967 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.091106 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1102 00:59:22.091192 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:22.091346 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:22.091451 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:22.091504 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.091666 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1102 00:59:22.094041 34543 executor.go:111] Tasks: 107 done / 116 total; 5 can run W1102 00:59:22.094199 34543 api.go:53] PageSize not implemented I1102 00:59:22.094228 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:22.094245 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1102 00:59:22.094269 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1102 00:59:22.094417 34543 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I1102 00:59:22.094333 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:22.094450 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W1102 00:59:22.094540 34543 api.go:53] PageSize not implemented I1102 00:59:22.094577 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1102 00:59:22.094805 34543 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1102 00:59:22.094959 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1102 00:59:22.095112 34543 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1102 00:59:22.098674 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:22.098751 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:22.098824 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:22.098892 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:22.098965 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:22.099035 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:22.099105 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:22.099167 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:22.099239 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:22.099304 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:22.099374 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1102 00:59:22.099443 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1102 00:59:22.099504 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:22.099559 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:22.099621 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:22.099690 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:22.099750 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1102 00:59:22.099814 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:22.099877 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:22.099939 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1102 00:59:22.100005 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1102 00:59:22.100066 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1102 00:59:22.100131 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:22.100193 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:22.100276 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1102 00:59:22.100343 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:22.100399 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:22.100460 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:22.100527 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:22.100593 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:22.102206 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:22.102243 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:22.102299 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102391 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1102 00:59:22.102497 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102582 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102641 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102708 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102770 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.102844 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1102 00:59:22.102929 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.103009 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:22.103058 34543 natgateway.go:117] DescribeNatGateways: { } I1102 00:59:22.103129 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1102 00:59:22.103184 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W1102 00:59:22.103264 34543 api.go:53] PageSize not implemented I1102 00:59:22.103294 34543 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1102 00:59:22.103379 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:22.103418 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:22.103447 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:22.103485 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:22.103514 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:22.103616 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.103669 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.103743 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:22.103805 34543 iamrole.go:89] ListRoles: { } I1102 00:59:22.103880 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:22.103951 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:22.104009 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1102 00:59:22.104067 34543 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1102 00:59:22.104128 34543 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1102 00:59:22.104748 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I1102 00:59:22.104783 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1102 00:59:22.104867 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1102 00:59:22.104877 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } autoscaling-group:bastion.private-shared-ip.example.com ok I1102 00:59:22.104931 34543 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } W1102 00:59:22.105001 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.104961 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } autoscaling-group:nodes.private-shared-ip.example.com ok iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I1102 00:59:22.105030 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok load-balancer:bastion-private-shared-ip-eepmph ok I1102 00:59:22.105068 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I1102 00:59:22.105040 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } keypair:key-1 ok I1102 00:59:22.105129 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I1102 00:59:22.104860 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.105126 34543 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok autoscaling-config:lt-2 ok nat-gateway:nat-1 ok I1102 00:59:22.105169 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } autoscaling-config:lt-3 ok I1102 00:59:22.105209 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-config:lt-1 ok I1102 00:59:22.105247 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I1102 00:59:22.105267 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1102 00:59:22.105424 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.105448 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1102 00:59:22.105485 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:22.105543 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1102 00:59:22.105750 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.105770 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:22.105801 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1102 00:59:22.105861 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:22.105961 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1102 00:59:22.105995 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I1102 00:59:22.106011 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I1102 00:59:22.106084 34543 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com W1102 00:59:22.106094 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.106102 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1102 00:59:22.106141 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I1102 00:59:22.106122 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1102 00:59:22.106175 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I1102 00:59:22.106255 34543 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1102 00:59:22.106277 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1102 00:59:22.106256 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:22.106315 34543 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I1102 00:59:22.106345 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I1102 00:59:22.106389 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1102 00:59:22.106402 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1102 00:59:22.106429 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I1102 00:59:22.106352 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W1102 00:59:22.106524 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.106556 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:22.106610 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:22.106701 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1102 00:59:22.106790 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:22.106897 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1102 00:59:22.107201 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.38s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I1102 00:59:22.107511 34543 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:59:22.107814 34543 internetgateways.go:66] CreateInternetGateway: { } I1102 00:59:22.107876 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:59:22.107944 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:59:22.108021 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:59:22.108078 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:59:22.108131 34543 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:59:22.108191 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:59:22.108240 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:59:22.108339 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I1102 00:59:22.133105 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:22.136377 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.136499 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.175082 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.311870 34543 iamrole.go:89] ListRoles: { } I1102 00:59:22.313701 34543 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1102 00:59:22.313976 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I1102 00:59:22.314084 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1102 00:59:22.314313 34543 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.314269 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.314433 34543 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1102 00:59:22.314642 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:22.314583 34543 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.314799 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:22.314877 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.315098 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:22.315251 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:22.315306 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.315367 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.315452 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.315751 34543 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.315768 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.315985 34543 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.316267 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1102 00:59:22.316328 34543 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1102 00:59:22.316383 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:22.316461 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.316535 34543 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1102 00:59:22.316749 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:22.316999 34543 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1102 00:59:22.317120 34543 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1102 00:59:22.317125 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.317340 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1102 00:59:22.317387 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.317411 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.317573 34543 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1102 00:59:22.317433 34543 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I1102 00:59:22.317559 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.317777 34543 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.317912 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:22.317968 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.318117 34543 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:59:22.318179 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1102 00:59:22.318258 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.318392 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1102 00:59:22.318745 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.318893 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1102 00:59:22.318963 34543 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1102 00:59:22.319052 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:22.319041 34543 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1102 00:59:22.319116 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:22.319163 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:22.319212 34543 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] } I1102 00:59:22.319306 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:22.319407 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:22.319455 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:22.319502 34543 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1102 00:59:22.319672 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:22.319765 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:22.320524 34543 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1102 00:59:22.322468 34543 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1102 00:59:22.322636 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.322771 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.323000 34543 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:22.323168 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.323358 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.323509 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.323768 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.324012 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.324223 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1102 00:59:22.324319 34543 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1102 00:59:22.324410 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.324576 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.324797 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.324993 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.325172 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.325341 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.325505 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.325679 34543 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1102 00:59:22.325749 34543 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.325915 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.326091 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I1102 00:59:22.326276 34543 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1102 00:59:22.326461 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.326541 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.326652 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.328935 34543 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1102 00:59:22.329222 34543 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1102 00:59:22.329481 34543 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1102 00:59:22.329533 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.329572 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.329631 34543 dns.go:238] Pre-creating DNS records I1102 00:59:22.329663 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.335780 34543 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1102 00:59:22.338947 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.339039 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.377704 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.507878 34543 iamrole.go:89] ListRoles: { } I1102 00:59:22.509696 34543 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1102 00:59:22.509942 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.510247 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.510390 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:59:22.510426 34543 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:59:22.510480 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.510520 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.510635 34543 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1102 00:59:22.510657 34543 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1102 00:59:22.510834 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.511011 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.512544 34543 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1102 00:59:22.512663 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.512786 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1102 00:59:22.513001 34543 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1102 00:59:22.512933 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1102 00:59:22.513051 34543 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1102 00:59:22.513103 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.513337 34543 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:59:22.513384 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.513520 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1102 00:59:22.513636 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.513778 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1102 00:59:22.513841 34543 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:59:22.517419 34543 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1102 00:59:22.517701 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.517856 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.518063 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.518229 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.518488 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.518615 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.518763 34543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1102 00:59:22.520994 34543 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1102 00:59:22.521290 34543 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1102 00:59:22.521442 34543 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1102 00:59:22.524292 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1102 00:59:22.524357 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1102 00:59:22.524393 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1102 00:59:22.524425 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1102 00:59:22.524459 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1102 00:59:22.524499 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1102 00:59:22.524535 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1102 00:59:22.524568 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1102 00:59:22.524601 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1102 00:59:22.524636 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1102 00:59:22.524671 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1102 00:59:22.524708 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1102 00:59:22.524751 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1102 00:59:22.524785 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1102 00:59:22.524817 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1102 00:59:22.524853 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1102 00:59:22.524885 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:59:22.524921 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1102 00:59:22.524956 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:59:22.524993 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1102 00:59:22.525025 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1102 00:59:22.525056 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1102 00:59:22.525088 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1102 00:59:22.525125 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1102 00:59:22.525163 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:59:22.525200 34543 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1102 00:59:22.526644 34543 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1102 00:59:22.526673 34543 keypairs.go:95] DescribeKeyPairs: { } I1102 00:59:22.526715 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.526767 34543 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.526824 34543 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.526902 34543 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.526952 34543 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527010 34543 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527053 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527094 34543 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527185 34543 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527242 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1102 00:59:22.527283 34543 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W1102 00:59:22.527341 34543 api.go:53] PageSize not implemented I1102 00:59:22.527364 34543 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1102 00:59:22.527391 34543 loadbalancers.go:34] PageSize not implemented I1102 00:59:22.527411 34543 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1102 00:59:22.527436 34543 targetgroups.go:35] PageSize not implemented I1102 00:59:22.527458 34543 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1102 00:59:22.527508 34543 zones.go:93] ListHostedZonesPages { } I1102 00:59:22.527538 34543 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.527588 34543 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1102 00:59:22.527624 34543 iamrole.go:89] ListRoles: { } I1102 00:59:22.527666 34543 internetgateways.go:101] DescribeInternetGateways: { } I1102 00:59:22.527711 34543 routetable.go:53] DescribeRouteTables: { } I1102 00:59:22.527745 34543 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1102 00:59:22.528189 34543 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1102 00:59:22.528260 34543 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1102 00:59:22.528242 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.528305 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I1102 00:59:22.528338 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I1102 00:59:22.528368 34543 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } volume:vol-2 ok autoscaling-config:lt-2 ok I1102 00:59:22.528417 34543 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1102 00:59:22.528394 34543 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-config:lt-1 ok I1102 00:59:22.528478 34543 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I1102 00:59:22.528480 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok keypair:key-1 ok I1102 00:59:22.528530 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1102 00:59:22.528656 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.528678 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1102 00:59:22.528462 34543 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1102 00:59:22.528710 34543 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I1102 00:59:22.528754 34543 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1102 00:59:22.528948 34543 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1102 00:59:22.528962 34543 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1102 00:59:22.529028 34543 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1102 00:59:22.529085 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I1102 00:59:22.529194 34543 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1102 00:59:22.529214 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1102 00:59:22.529257 34543 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I1102 00:59:22.529293 34543 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.529341 34543 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1102 00:59:22.529354 34543 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1102 00:59:22.529383 34543 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I1102 00:59:22.529443 34543 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1102 00:59:22.529522 34543 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1102 00:59:22.529600 34543 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1102 00:59:22.529933 34543 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 62.373s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.025s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I1102 00:58:00.219028 25701 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.042s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.036s === RUN TestPodController I1102 00:58:05.223988 30152 pod.go:59] starting pod controller I1102 00:58:05.224313 30152 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.052s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.046s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00010fdf0} route53_test.go:93: Got zone list: [0xc00010fe20] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00010fef0} route53_test.go:93: Got zone list: [0xc00010ff20] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00010ffc0} route53_test.go:170: Successfully added managed DNS zone: &{0xc00032f500 0xc0001260a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352120} route53_test.go:93: Got zone list: [0xc000352150] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc0001260c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352250} route53_test.go:93: Got zone list: [0xc000352280] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc0001260e0} route53_test.go:187: Successfully added resource record set: {0xc000184600 0xc000126100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352500} route53_test.go:93: Got zone list: [0xc000352530] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126188} route53_test.go:199: Successfully added resource record set: {0xc000184780 0xc0001261a8} route53_test.go:118: Got 1 recordsets: [0xc0003526d0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352800} route53_test.go:93: Got zone list: [0xc000352830] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126240} route53_test.go:221: Successfully added resource record set: {0xc000184900 0xc000126260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000184900 0xc000126260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352b90} route53_test.go:93: Got zone list: [0xc000352bc0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126318} route53_test.go:246: Successfully removed resource set {0xc000184b00 0xc000126338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000352e40} route53_test.go:93: Got zone list: [0xc000352e70] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc0001263c0} route53_test.go:264: Successfully removed resource set {0xc000184c80 0xc0001263e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000353120} route53_test.go:93: Got zone list: [0xc000353150] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126470} commontests.go:192: Successfully added resource record set: {0xc000184e00 0xc000126490} commontests.go:45: Correctly replaced resource record {0xc000184e00 0xc000126490} -> {0xc000184f00 0xc0001264e0} commontests.go:142: Got 1 recordsets: [0xc000353460] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000353630} route53_test.go:93: Got zone list: [0xc000353660] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc000126590} commontests.go:192: Successfully added resource record set: {0xc000185180 0xc0001265b0} commontests.go:70: Correctly replaced resource record {0xc000185180 0xc0001265b0} -> {0xc000185280 0xc000126600} commontests.go:142: Got 1 recordsets: [0xc000353970] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000353b90} route53_test.go:93: Got zone list: [0xc000353bc0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc0001266c0} commontests.go:192: Successfully added resource record set: {0xc000185500 0xc0001266e0} commontests.go:142: Got 2 recordsets: [0xc000353e30 0xc000353e50] commontests.go:142: Got 2 recordsets: [0xc000353ec0 0xc000353ee0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000364080} route53_test.go:93: Got zone list: [0xc0003640b0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032f320 0xc0001267e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.051s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.058s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca380} clouddns_test.go:71: Got zone list: [0xc000129038] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000128fd8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca480} clouddns_test.go:71: Got zone list: [0xc0001291d0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129140} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca580} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0003802c0 0xc000129320} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca640} clouddns_test.go:71: Got zone list: [0xc000129470] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129410} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca760} clouddns_test.go:71: Got zone list: [0xc0001296b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129680} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000ca8a0} clouddns_test.go:71: Got zone list: [0xc000129800] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc0001297d0} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000caa00} clouddns_test.go:71: Got zone list: [0xc0001299f8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc0001299c8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000cab60} clouddns_test.go:71: Got zone list: [0xc000129b78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129b48} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000caca0} clouddns_test.go:71: Got zone list: [0xc000129cc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129c98} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000cae00} clouddns_test.go:71: Got zone list: [0xc000129e78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc000129e48} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000caf60} clouddns_test.go:71: Got zone list: [0xc0001a21f8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc0001a21c8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.01s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000cb0e0} clouddns_test.go:71: Got zone list: [0xc0001a2510] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc0001a24e0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a078 0xc0000cb2c0} clouddns_test.go:71: Got zone list: [0xc0001a2a38] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000380180 0xc0001a2a08} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.099s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.032s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.094s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I1102 00:58:19.762508 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.762742 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.762790 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.762822 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.762861 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.762889 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.762904 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.762942 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.762968 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.769262 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I1102 00:58:19.772826 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.772997 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.773038 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.773060 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.773090 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.773121 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.773134 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.773153 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.773165 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.774047 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I1102 00:58:19.776713 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.776859 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.776888 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.776909 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.776937 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.776953 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.776967 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.776987 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.776999 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.777788 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.778889 34478 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I1102 00:58:19.778904 34478 assetstore.go:111] bin/containerd I1102 00:58:19.778911 34478 assetstore.go:111] bin/containerd-shim I1102 00:58:19.778916 34478 assetstore.go:111] bin/containerd-shim-runc-v1 I1102 00:58:19.778922 34478 assetstore.go:111] bin/containerd-shim-runc-v2 I1102 00:58:19.778927 34478 assetstore.go:111] bin/containerd-stress I1102 00:58:19.778933 34478 assetstore.go:111] bin/ctr I1102 00:58:19.778963 34478 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I1102 00:58:19.778974 34478 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I1102 00:58:19.782767 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.782933 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.782979 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.783007 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.783034 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.783058 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.783085 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.783106 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.783119 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.784126 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.785371 34478 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I1102 00:58:19.786197 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.786319 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.786340 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.786360 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.786382 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.786438 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.786453 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.786470 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.786483 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.787286 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.788461 34478 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I1102 00:58:19.789010 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.789125 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.789149 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.789173 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.789195 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.789230 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.789247 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.789266 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.789278 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.790077 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.791344 34478 assetstore.go:108] Matching assets for "^docker/": I1102 00:58:19.791360 34478 assetstore.go:111] docker/containerd I1102 00:58:19.791367 34478 assetstore.go:111] docker/containerd-shim I1102 00:58:19.791373 34478 assetstore.go:111] docker/ctr I1102 00:58:19.791378 34478 assetstore.go:111] docker/docker I1102 00:58:19.791383 34478 assetstore.go:111] docker/docker-init I1102 00:58:19.791389 34478 assetstore.go:111] docker/docker-proxy I1102 00:58:19.791396 34478 assetstore.go:111] docker/dockerd I1102 00:58:19.791403 34478 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I1102 00:58:19.793860 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.794015 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.794041 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.794066 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.794091 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.794132 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.794148 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.794168 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.794181 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.795093 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.796298 34478 assetstore.go:108] Matching assets for "^docker/": I1102 00:58:19.796313 34478 assetstore.go:111] docker/containerd I1102 00:58:19.796320 34478 assetstore.go:111] docker/containerd-shim I1102 00:58:19.796326 34478 assetstore.go:111] docker/ctr I1102 00:58:19.796332 34478 assetstore.go:111] docker/docker I1102 00:58:19.796337 34478 assetstore.go:111] docker/docker-init I1102 00:58:19.796342 34478 assetstore.go:111] docker/docker-proxy I1102 00:58:19.796347 34478 assetstore.go:111] docker/dockerd I1102 00:58:19.796353 34478 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.00s) === RUN TestDockerBuilder_SkipInstall I1102 00:58:19.798647 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.798793 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.798820 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.798846 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.798872 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.798914 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.798930 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.798950 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.798965 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.799820 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.801025 34478 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I1102 00:58:19.801166 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.801272 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.801296 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.801316 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.801338 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.801373 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.801388 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.801444 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.801457 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.802242 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:19.803674 34478 assetstore.go:108] Matching assets for "^docker/": I1102 00:58:19.803689 34478 assetstore.go:111] docker/docker I1102 00:58:19.803696 34478 assetstore.go:111] docker/docker-containerd I1102 00:58:19.803702 34478 assetstore.go:111] docker/docker-containerd-ctr I1102 00:58:19.803707 34478 assetstore.go:111] docker/docker-containerd-shim I1102 00:58:19.803713 34478 assetstore.go:111] docker/docker-init I1102 00:58:19.803719 34478 assetstore.go:111] docker/docker-proxy I1102 00:58:19.803724 34478 assetstore.go:111] docker/docker-runc I1102 00:58:19.803731 34478 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I1102 00:58:19.807122 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.807257 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.807285 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.807321 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.807345 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.807367 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.807381 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.807404 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.807420 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.809613 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I1102 00:58:19.811657 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.811773 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.811813 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.811840 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.811866 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.811891 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.811906 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.811922 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.811938 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.815649 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I1102 00:58:19.819099 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.819329 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.819368 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.819405 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.819444 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.819477 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.819502 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.819532 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.819556 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.822103 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I1102 00:58:19.830799 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.831050 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.831088 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.831127 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.831168 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.831203 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.831230 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.831263 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.831285 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.833850 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I1102 00:58:19.846672 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.846912 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.846961 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.846997 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.847037 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.847070 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.847096 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.847128 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.847149 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.849582 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I1102 00:58:19.858828 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.859034 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.859071 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.859106 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.859151 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.859183 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.859207 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.859237 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.859258 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.862583 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I1102 00:58:19.868156 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.868327 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.868354 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.868380 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.868406 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.868426 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.868440 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.868459 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.868474 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.869951 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I1102 00:58:19.876256 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:19.876415 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:19.876444 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:19.876483 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:19.876511 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:19.876534 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:19.876551 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:19.876573 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:19.876587 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:19.878091 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.38s) === RUN TestKubeAPIServerBuilderARM64 I1102 00:58:21.253960 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.254285 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.254329 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.254358 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.254386 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.254412 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.254426 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.254445 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.254460 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.256144 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.07s) === RUN TestKubeControllerManagerBuilder I1102 00:58:21.322994 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.323149 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.323189 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.323215 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.323241 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.323274 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.323287 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.323307 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.323321 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.324803 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I1102 00:58:21.329829 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.329959 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.329987 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.330011 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.330036 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.330058 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.330074 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.330095 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.330108 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.331483 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I1102 00:58:21.398142 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.398373 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.398417 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.398461 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.398500 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.398533 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.398554 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.398585 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.398605 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.400842 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I1102 00:58:21.465860 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.466038 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.466078 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.466103 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.466131 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.466164 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.466184 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.466204 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.466237 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.467659 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.00s) === RUN TestKubeProxyBuilderAMD64 I1102 00:58:21.470854 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.471029 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.471059 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.471082 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.471107 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.471129 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.471144 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.471165 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.471179 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.472666 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I1102 00:58:21.535925 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.536085 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.536114 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.536139 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.536165 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.536188 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.536203 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.536222 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.536234 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.537684 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I1102 00:58:21.607355 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.607602 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.607658 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.607700 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.607744 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.607781 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.607807 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.607841 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.607865 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.610357 34478 zones.go:93] ListHostedZonesPages { } W1102 00:58:21.615858 34478 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I1102 00:58:21.618367 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.618593 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.618642 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.618677 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.618712 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.618748 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.618771 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.618805 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.618824 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.620977 34478 zones.go:93] ListHostedZonesPages { } W1102 00:58:21.684316 34478 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I1102 00:58:21.686618 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.686786 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.686819 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.686853 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.686911 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.686951 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.686973 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.687014 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.687033 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.688886 34478 zones.go:93] ListHostedZonesPages { } W1102 00:58:21.752812 34478 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I1102 00:58:21.754672 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.754845 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.754881 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.754911 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.754944 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.754973 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.754989 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.755012 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.755030 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.756712 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.758410 34478 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I1102 00:58:21.760390 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.760516 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.760557 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.760581 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.760606 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.760631 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.760648 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.760670 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.760686 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.761968 34478 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I1102 00:58:21.767272 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.767428 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.767475 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.767505 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.767534 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.767560 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.767577 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.767599 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.767614 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.768960 34478 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I1102 00:58:21.772182 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.772321 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.772348 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.772370 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.772396 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.772417 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.772434 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.772454 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.772466 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.774124 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.775686 34478 assetstore.go:108] Matching assets for "protokube$": I1102 00:58:21.775705 34478 assetstore.go:111] /opt/kops/bin/protokube I1102 00:58:21.775725 34478 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I1102 00:58:21.775745 34478 assetstore.go:108] Matching assets for "channels$": I1102 00:58:21.775753 34478 assetstore.go:111] /opt/kops/bin/channels I1102 00:58:21.775761 34478 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I1102 00:58:21.777111 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.777245 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.777272 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.777297 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.777324 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.777345 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.777367 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.777388 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.777403 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.778982 34478 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I1102 00:58:21.781044 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.781170 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.781196 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.781218 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.781243 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.781264 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.781281 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.781302 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.781314 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.783011 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.784497 34478 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I1102 00:58:21.784906 34478 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:21.785032 34478 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.785057 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:21.785080 34478 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:21.785103 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:21.785124 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:21.785142 34478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:21.785162 34478 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:21.785176 34478 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:21.786668 34478 zones.go:93] ListHostedZonesPages { } I1102 00:58:21.788149 34478 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.089s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.083s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.061s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.059s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.106s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E1102 00:58:03.697490 28692 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E1102 00:58:03.697597 28692 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.057s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I1102 00:58:13.934611 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.934714 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.934750 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.934778 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.934806 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I1102 00:58:13.935054 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935094 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935120 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935148 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935181 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935207 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935247 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935304 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935340 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935370 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935415 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935489 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935520 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935545 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935570 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935608 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935635 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935692 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1102 00:58:13.935722 32732 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I1102 00:58:13.935912 32732 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I1102 00:58:13.935967 32732 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.153s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I1102 00:58:06.413014 30690 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1102 00:58:06.413899 30690 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1102 00:58:06.415332 30690 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1102 00:58:06.415691 30690 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1102 00:58:06.418147 30690 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1102 00:58:06.418719 30690 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1102 00:58:06.419578 30690 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1102 00:58:06.420163 30690 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1102 00:58:06.421095 30690 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I1102 00:58:06.421201 30690 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I1102 00:58:06.421662 30690 health.go:33] status conditions not found for /v1, Kind=Namespace I1102 00:58:06.422049 30690 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I1102 00:58:06.422144 30690 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I1102 00:58:06.422520 30690 health.go:33] status conditions not found for /v1, Kind=ConfigMap I1102 00:58:06.422933 30690 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I1102 00:58:06.423070 30690 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I1102 00:58:06.423995 30690 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I1102 00:58:06.424054 30690 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.06s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.102s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W1102 00:58:04.481539 28781 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.66s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W1102 00:58:04.482575 28781 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W1102 00:58:04.483629 28781 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W1102 00:58:04.484645 28781 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W1102 00:58:04.485647 28781 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W1102 00:58:04.486673 28781 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W1102 00:58:04.487747 28781 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E1102 00:58:04.488734 28781 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E1102 00:58:04.488746 28781 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E1102 00:58:04.488759 28781 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E1102 00:58:04.488770 28781 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.772s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.030s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.136s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.101s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.165s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.047s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.053s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.060s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.03s) PASS ok k8s.io/kops/pkg/edit 0.105s === RUN TestFlagToFalse I1102 00:58:05.618191 30459 featureflag_test.go:32] Created flag Unittest1 I1102 00:58:05.618248 30459 featureflag.go:160] FeatureFlag "UnitTest1"=false I1102 00:58:05.618261 30459 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I1102 00:58:05.618323 30459 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.021s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.01s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.02s) PASS ok k8s.io/kops/pkg/flagbuilder 0.104s === RUN TestRenderInstanceGroupZones W1102 00:58:08.328208 31198 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.045s === RUN TestWarmPoolOnlyRoll I1102 00:58:31.464548 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.464608 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.464620 34929 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I1102 00:58:31.464970 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.464986 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.466081 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.466130 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.471431 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.471464 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.471482 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.471493 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.473665 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.474458 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.474491 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.475705 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.475751 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.481059 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.481118 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.481949 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.481988 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.483193 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.483228 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.488514 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.488554 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.488575 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.488590 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.489625 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.490440 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.490476 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.491658 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.491693 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.496946 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.496997 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.497818 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.497855 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.499059 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.499094 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.504373 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.504424 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.505351 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.505387 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.506604 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.506636 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.511891 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.511924 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.511945 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.511961 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.512945 34929 instancegroups.go:402] Draining the node: "node-2a.local". I1102 00:58:31.513721 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.513758 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.514944 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.514979 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.520248 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.520297 34929 instancegroups.go:402] Draining the node: "node-2b.local". I1102 00:58:31.521120 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.521158 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.522360 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.522414 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.527643 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.527695 34929 instancegroups.go:402] Draining the node: "node-2c.local". I1102 00:58:31.528530 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.528570 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.529774 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.529809 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.535054 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.535088 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I1102 00:58:31.535628 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.535663 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.536855 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.536931 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.536980 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.536998 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.537027 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.538222 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.538273 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.538288 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.538316 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.539511 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.539546 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.539595 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.539610 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.539636 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.540837 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.540888 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.540903 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.540931 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.542133 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.542183 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.542197 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.542224 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.543428 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.543454 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.543503 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.543520 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.543561 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.544790 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.544841 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.544856 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.544883 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.546080 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1102 00:58:31.546131 34929 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1102 00:58:31.546145 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.546172 34929 instancegroups.go:438] waiting for 1ms after terminating instance W1102 00:58:31.547356 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1102 00:58:31.547389 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I1102 00:58:31.547778 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.547808 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.549002 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.549044 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.549083 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.549137 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.549181 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.549196 34929 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.549208 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.549928 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.550785 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.550827 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.552029 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.552077 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.552099 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.552140 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.552972 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.553012 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.554201 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.554243 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.554265 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.554286 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.554307 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.554321 34929 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.554334 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.555332 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.556161 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.556200 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.557388 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.557430 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.557452 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.557493 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.558400 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.558440 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.559633 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.559674 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.559696 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.559736 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.560615 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.560654 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.561861 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.561903 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.561925 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.561946 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.561967 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.561981 34929 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.561995 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.562966 34929 instancegroups.go:402] Draining the node: "node-2a.local". I1102 00:58:31.563732 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.563772 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.564989 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.565030 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.565051 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.565091 34929 instancegroups.go:402] Draining the node: "node-2b.local". I1102 00:58:31.565901 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.565940 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.567121 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.567163 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.567184 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.567225 34929 instancegroups.go:402] Draining the node: "node-2c.local". I1102 00:58:31.568063 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.568102 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.569304 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.569346 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1102 00:58:31.569367 34929 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1102 00:58:31.569385 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I1102 00:58:31.569659 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I1102 00:58:31.569945 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.569977 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.571177 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.571214 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.576498 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.576559 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.576605 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.576622 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.577411 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.578286 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.578325 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.579528 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.579563 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.584870 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.584954 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.586229 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.586302 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.587536 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.587571 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.592810 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.592850 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.592870 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.592896 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.593983 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.594829 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.594867 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.596063 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.596097 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.601340 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.601391 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.602237 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.602277 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.603471 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.603505 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.608771 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.608850 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.609756 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.609797 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.611004 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.611040 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.616283 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.616315 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.616336 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.616349 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.617353 34929 instancegroups.go:402] Draining the node: "node-2a.local". I1102 00:58:31.618132 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.618186 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.619390 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.619425 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.624690 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.624752 34929 instancegroups.go:402] Draining the node: "node-2b.local". I1102 00:58:31.625757 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.625797 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.626987 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.627046 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.632354 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.632406 34929 instancegroups.go:402] Draining the node: "node-2c.local". I1102 00:58:31.633343 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.633382 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.634575 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.634607 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.639854 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.639888 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I1102 00:58:31.640046 34929 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I1102 00:58:31.640617 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.640650 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.641845 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.641887 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.641907 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I1102 00:58:31.642424 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.642460 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.643647 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.643685 34929 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1102 00:58:31.643707 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I1102 00:58:31.643977 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.644014 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.644030 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I1102 00:58:31.644227 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.644254 34929 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1102 00:58:31.644270 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I1102 00:58:31.644533 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.644567 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.645755 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.645796 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.651062 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.651123 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.651148 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.651162 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.652025 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.652966 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.653008 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.654190 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.654237 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.654278 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I1102 00:58:31.654663 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.654697 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.655893 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.655934 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.661202 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.661262 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.661286 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.661299 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.662151 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.663052 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.663092 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.664297 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.664359 34929 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1102 00:58:31.664385 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I1102 00:58:31.664704 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.664731 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.664744 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.665915 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.666846 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.666900 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.668106 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.668162 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.673489 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.673555 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.674506 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.674546 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.675753 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.675789 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.681062 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.681125 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.682090 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.682129 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.683337 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.683384 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.688617 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.688652 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I1102 00:58:31.689071 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.689112 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.689133 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I1102 00:58:31.689488 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.689527 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.689549 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I1102 00:58:31.689888 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.689919 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.689936 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I1102 00:58:31.690152 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.690183 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.690212 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.691706 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.692819 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.692857 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.694050 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.694097 34929 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1102 00:58:31.694130 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I1102 00:58:31.694387 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.694419 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.694433 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.695616 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.696476 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.696514 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.697697 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.697740 34929 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1102 00:58:31.697762 34929 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I1102 00:58:31.698120 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.698157 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.699328 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.699376 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.700530 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.705809 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.706962 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.712238 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.713422 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.718696 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.718783 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.718813 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.718833 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.719709 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.720861 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.720903 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.722127 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.722177 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.727446 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.727516 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.728521 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.728561 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.729748 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.729806 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.735146 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.735187 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.735217 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.735231 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.736420 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.737378 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.737418 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.738635 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.738689 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.743960 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.744031 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.746673 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.747056 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.748332 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.748400 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.753655 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.753723 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.754687 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.754719 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.755943 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.756003 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.761328 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.761358 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.761383 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.761397 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.762364 34929 instancegroups.go:402] Draining the node: "node-2a.local". I1102 00:58:31.763066 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.763097 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.764324 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.764374 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.769670 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.769716 34929 instancegroups.go:402] Draining the node: "node-2b.local". I1102 00:58:31.770512 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.770545 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.771763 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.771818 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.777152 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.777205 34929 instancegroups.go:402] Draining the node: "node-2c.local". I1102 00:58:31.778092 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.778128 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.779350 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.779401 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.784721 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.784756 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I1102 00:58:31.785116 34929 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1102 00:58:31.785148 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.786362 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.786407 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.787574 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.788786 34929 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1102 00:58:31.789963 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.795236 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.795305 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.795335 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.795354 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.796244 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.797216 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.797259 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.798489 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.798526 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.803926 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.803982 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.804928 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.804969 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.806212 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.806247 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.811470 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.811511 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.811534 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.811549 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.812696 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.813618 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.813659 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.814911 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.814948 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.820250 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.820305 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.821259 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.821299 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.822542 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.822578 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.827858 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.827914 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.828869 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.828910 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.830152 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.830187 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.835484 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.835518 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.835540 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.835555 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.836746 34929 instancegroups.go:402] Draining the node: "node-2a.local". I1102 00:58:31.837651 34929 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1102 00:58:31.837692 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.838953 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.838997 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.844264 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.844318 34929 instancegroups.go:402] Draining the node: "node-2b.local". I1102 00:58:31.845250 34929 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1102 00:58:31.845290 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.846522 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.846558 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.851830 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.851888 34929 instancegroups.go:402] Draining the node: "node-2c.local". I1102 00:58:31.852774 34929 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1102 00:58:31.852815 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.854027 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.854063 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.859366 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.859402 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I1102 00:58:31.860011 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.860039 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.860051 34929 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1102 00:58:31.860825 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.861602 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.861642 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.862859 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.862905 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.868204 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.868256 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.869135 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.869174 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.870385 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.870421 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.875673 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.875708 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I1102 00:58:31.875962 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.875990 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.876003 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.876890 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:31.877656 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:31.877696 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.878942 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.878976 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.884255 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.884307 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:31.885211 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:31.885252 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.886498 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.886532 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.891785 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.891819 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I1102 00:58:31.892155 34929 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled W1102 00:58:31.892198 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1102 00:58:31.892225 34929 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled W1102 00:58:31.892242 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1102 00:58:31.892263 34929 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled W1102 00:58:31.892273 34929 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1102 00:58:31.892292 34929 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1102 00:58:31.892317 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I1102 00:58:31.892559 34929 instancegroups.go:572] Detaching instance "bastion-1a", in group "bastion-1". I1102 00:58:31.892616 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:31.893820 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.893856 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.899109 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.899149 34929 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled I1102 00:58:31.899194 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.899215 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.899228 34929 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1102 00:58:31.899930 34929 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled I1102 00:58:31.899962 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.900005 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.900021 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:31.900934 34929 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1102 00:58:31.900996 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:31.902229 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.902264 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.907540 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.907579 34929 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled I1102 00:58:31.907595 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.907614 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.907627 34929 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1102 00:58:31.908550 34929 instancegroups.go:570] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I1102 00:58:31.908610 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:31.909847 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.909882 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:31.915161 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.915208 34929 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1102 00:58:31.915228 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I1102 00:58:31.915450 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.915487 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.915501 34929 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1102 00:58:31.917628 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:31.918384 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:31.918430 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.919668 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.919715 34929 instancegroups.go:505] Cluster validated. I1102 00:58:31.919760 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:31.919961 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:31.921417 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:31.921763 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:31.921809 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:31.923041 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:31.923116 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.023726 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.023781 34929 instancegroups.go:402] Draining the node: "node-1e.local". I1102 00:58:32.023819 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:32.025221 34929 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1102 00:58:32.025500 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:32.025543 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.026757 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.026802 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.026843 34929 instancegroups.go:402] Draining the node: "node-1f.local". I1102 00:58:32.027706 34929 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1102 00:58:32.027755 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.029010 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.029073 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.129697 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.129747 34929 instancegroups.go:402] Draining the node: "node-1g.local". I1102 00:58:32.130621 34929 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1102 00:58:32.130666 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.131886 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.131931 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.131948 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I1102 00:58:32.132181 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.132216 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.132230 34929 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1102 00:58:32.134090 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:32.134180 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:32.134919 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:32.135563 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:32.135608 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.136851 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.136936 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.237531 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.237585 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:32.237615 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:32.239012 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:32.239316 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:32.239359 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.240552 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.240599 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.240647 34929 instancegroups.go:402] Draining the node: "node-1e.local". I1102 00:58:32.241592 34929 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1102 00:58:32.241640 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.242882 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.242944 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.343559 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.343618 34929 instancegroups.go:402] Draining the node: "node-1f.local". I1102 00:58:32.344561 34929 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1102 00:58:32.344605 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.345825 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.345870 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.345887 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I1102 00:58:32.346166 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.346202 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.346228 34929 instancegroups.go:313] Tainting 7 nodes in "master-1" instancegroup. I1102 00:58:32.348852 34929 instancegroups.go:402] Draining the node: "master-1a.local". I1102 00:58:32.349742 34929 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1102 00:58:32.349792 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.351041 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.351086 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.351131 34929 instancegroups.go:402] Draining the node: "master-1c.local". I1102 00:58:32.351204 34929 instancegroups.go:402] Draining the node: "master-1b.local". I1102 00:58:32.352508 34929 instancegroups.go:593] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I1102 00:58:32.352887 34929 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1102 00:58:32.352939 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.354138 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.354210 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.454870 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.454927 34929 instancegroups.go:402] Draining the node: "master-1e.local". I1102 00:58:32.455024 34929 instancegroups.go:402] Draining the node: "master-1d.local". I1102 00:58:32.456306 34929 instancegroups.go:593] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I1102 00:58:32.456590 34929 instancegroups.go:593] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I1102 00:58:32.456640 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.457875 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.457919 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.457963 34929 instancegroups.go:402] Draining the node: "master-1f.local". I1102 00:58:32.458799 34929 instancegroups.go:593] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I1102 00:58:32.458843 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.460061 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.460125 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.560675 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.560727 34929 instancegroups.go:402] Draining the node: "master-1g.local". I1102 00:58:32.561556 34929 instancegroups.go:593] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I1102 00:58:32.561603 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.562841 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.562885 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.562903 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I1102 00:58:32.563155 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.563192 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.563206 34929 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1102 00:58:32.565125 34929 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1102 00:58:32.565194 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:32.566416 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.566463 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.566477 34929 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1102 00:58:32.566542 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:32.567776 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.567822 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.567870 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:32.567945 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:32.569232 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:32.569554 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:32.569605 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.570821 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.570886 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.671483 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.671539 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:32.671577 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:32.672889 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:32.673172 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:32.673216 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.674453 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.674498 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.674538 34929 instancegroups.go:402] Draining the node: "node-1e.local". I1102 00:58:32.675390 34929 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1102 00:58:32.675443 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.676663 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.676726 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.777334 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.777387 34929 instancegroups.go:402] Draining the node: "node-1f.local". I1102 00:58:32.778217 34929 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1102 00:58:32.778268 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.779513 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.779557 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.779575 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I1102 00:58:32.779859 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.779893 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.779907 34929 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1102 00:58:32.781746 34929 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1102 00:58:32.781793 34929 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1102 00:58:32.781839 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:32.783067 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.783112 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.783159 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:32.783229 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:32.784520 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:32.784842 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:32.784895 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.786157 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.786226 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.886821 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.886901 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:32.886923 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:32.888275 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:32.888610 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:32.888663 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.889873 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.889916 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.889958 34929 instancegroups.go:402] Draining the node: "node-1e.local". I1102 00:58:32.890849 34929 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1102 00:58:32.890902 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.892116 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.892185 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.993262 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.993339 34929 instancegroups.go:402] Draining the node: "node-1f.local". I1102 00:58:32.994278 34929 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1102 00:58:32.994336 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:32.995572 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.995629 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.995648 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I1102 00:58:32.995891 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.995918 34929 instancegroups.go:505] Cluster validated. I1102 00:58:32.995931 34929 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1102 00:58:32.996649 34929 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1102 00:58:32.996682 34929 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I1102 00:58:32.996720 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:32.997935 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:32.997973 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.003225 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.003281 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:33.003315 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:33.004686 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:33.004728 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.004992 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:33.005025 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.005947 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.005983 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.011290 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.011336 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I1102 00:58:33.011550 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.011577 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.011589 34929 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1102 00:58:33.012368 34929 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E1102 00:58:33.012428 34929 instancegroups.go:162] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I1102 00:58:33.012441 34929 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E1102 00:58:33.012484 34929 instancegroups.go:162] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I1102 00:58:33.012525 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:33.013711 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.013748 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.019016 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.019080 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:33.019163 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:33.020478 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:33.020533 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.021360 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:33.021398 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.021930 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.021980 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.027523 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.027558 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I1102 00:58:33.027826 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.027860 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.027873 34929 instancegroups.go:313] Tainting 4 nodes in "node-1" instancegroup. I1102 00:58:33.029356 34929 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1102 00:58:33.029430 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:33.030825 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.030888 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.030913 34929 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1102 00:58:33.031006 34929 instancegroups.go:174] waiting for 1ms after detaching instance I1102 00:58:33.032242 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.032288 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.032336 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:33.032421 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:33.032424 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:33.036030 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:33.036088 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.036385 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:33.036437 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.036797 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:33.036850 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.037304 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.138132 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.138222 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:33.139245 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:33.139299 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.140525 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.140592 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.140614 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I1102 00:58:33.140930 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.140965 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.140981 34929 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1102 00:58:33.143529 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:33.144442 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:33.144489 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.145697 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.145743 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.145792 34929 instancegroups.go:402] Draining the node: "node-1e.local". I1102 00:58:33.145945 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:33.147337 34929 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1102 00:58:33.147675 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:33.147729 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.148981 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.149073 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.249954 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.250023 34929 instancegroups.go:402] Draining the node: "node-1g.local". I1102 00:58:33.250102 34929 instancegroups.go:402] Draining the node: "node-1f.local". I1102 00:58:33.251718 34929 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1102 00:58:33.252103 34929 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1102 00:58:33.252158 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.253392 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.253437 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.253477 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:33.254405 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:33.254452 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.255645 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.255711 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.356474 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.356564 34929 instancegroups.go:402] Draining the node: "node-1d.local". I1102 00:58:33.357671 34929 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1102 00:58:33.357729 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.358946 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.358995 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.359014 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I1102 00:58:33.359310 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.359339 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.359355 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1102 00:58:33.359374 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1102 00:58:33.359392 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1102 00:58:33.359412 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I1102 00:58:33.359571 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.359599 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.359612 34929 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1102 00:58:33.360755 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1102 00:58:33.360789 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1102 00:58:33.360806 34929 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1102 00:58:33.360850 34929 instancegroups.go:402] Draining the node: "node-1a.local". I1102 00:58:33.361692 34929 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1102 00:58:33.361730 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.362917 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.362955 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.368189 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.368249 34929 instancegroups.go:402] Draining the node: "node-1b.local". I1102 00:58:33.369192 34929 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1102 00:58:33.369230 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.370391 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.370426 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.375658 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.375707 34929 instancegroups.go:402] Draining the node: "node-1c.local". I1102 00:58:33.376632 34929 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1102 00:58:33.376671 34929 instancegroups.go:438] waiting for 1ms after terminating instance I1102 00:58:33.377880 34929 instancegroups.go:472] Validating the cluster. I1102 00:58:33.377912 34929 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1102 00:58:33.383152 34929 instancegroups.go:505] Cluster validated. I1102 00:58:33.383186 34929 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.990s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.035s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.033s === RUN TestParse E1102 00:58:06.855495 30753 versions.go:96] unable to parse Kubernetes version "" E1102 00:58:06.855597 30753 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.031s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.01s) PASS ok k8s.io/kops/pkg/kopscodecs 0.067s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.17s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.04s) PASS ok k8s.io/kops/pkg/kubeconfig 0.353s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.095s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.109s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.126s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I1102 00:58:13.935491 32734 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.135s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I1102 00:58:17.997170 33953 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W1102 00:58:18.041928 33953 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W1102 00:58:18.043853 33953 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.249s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I1102 00:58:18.007678 33968 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.06s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.05s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.208s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.05s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.04s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.202s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.036s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.238s === RUN TestServerGroupBuilder I1102 00:58:18.660024 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.676168 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.693590 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.713219 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.781408 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.804615 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.831870 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.850060 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.860776 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.921475 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.928579 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.937836 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.946423 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.952806 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:18.964286 34142 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.31s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.431s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.079s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.018s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.071s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (2.63s) --- PASS: TestIssueCert/ca (0.71s) --- PASS: TestIssueCert/client (0.97s) --- PASS: TestIssueCert/clientOneYear (0.93s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 2.664s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.009s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I1102 00:58:20.187189 34521 routetable.go:53] DescribeRouteTables: { } W1102 00:58:20.187326 34521 tags.go:67] cluster tag not found on route-table:rtb-1234 I1102 00:58:20.187342 34521 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I1102 00:58:20.187457 34521 iaminstanceprofile.go:193] ListInstanceProfiles: { } W1102 00:58:20.187535 34521 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I1102 00:58:20.187623 34521 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I1102 00:58:20.187724 34521 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I1102 00:58:20.187811 34521 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I1102 00:58:20.187897 34521 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I1102 00:58:20.188001 34521 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I1102 00:58:20.188051 34521 volumes.go:128] DescribeVolumes: { } W1102 00:58:20.188089 34521 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I1102 00:58:20.188245 34521 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:20.188468 34521 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:20.188515 34521 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:20.188599 34521 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:20.188661 34521 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:20.188709 34521 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:20.188729 34521 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:20.188759 34521 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:20.188783 34521 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:20.188806 34521 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I1102 00:58:20.188843 34521 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I1102 00:58:20.188870 34521 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I1102 00:58:20.188907 34521 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I1102 00:58:20.188931 34521 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I1102 00:58:20.188961 34521 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I1102 00:58:20.189005 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I1102 00:58:20.189041 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I1102 00:58:20.189073 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I1102 00:58:20.189105 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I1102 00:58:20.189138 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I1102 00:58:20.189168 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I1102 00:58:20.189202 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I1102 00:58:20.189229 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I1102 00:58:20.189264 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I1102 00:58:20.189293 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I1102 00:58:20.189324 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I1102 00:58:20.189352 34521 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.056s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.055s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.112s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.043s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.071s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.030s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.037s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.02s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.084s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.067s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.037s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W1102 00:58:11.284758 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284827 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284838 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284849 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284859 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284869 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284879 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284896 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284906 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284916 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284926 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284937 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284947 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284957 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.284968 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.284979 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.287563 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.287610 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.287703 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.287721 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.287787 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.287804 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.287869 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.287886 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.287953 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.287970 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288053 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288071 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288136 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288152 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288226 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288243 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288309 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288327 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288410 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288428 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288512 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288538 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288606 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288624 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288686 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288702 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288764 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288780 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288847 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288864 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.288931 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.288948 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289007 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289027 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289089 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289105 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289165 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289180 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289286 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289302 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289365 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289380 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289441 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289456 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289525 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289549 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289614 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289630 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289691 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289721 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289782 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289797 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289859 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289875 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.289933 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.289947 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290004 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290022 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290092 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290107 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290165 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290180 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290261 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290278 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290347 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290363 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290431 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290446 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290506 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290521 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290594 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290609 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290676 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290692 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290779 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290795 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290860 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290875 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.290941 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.290956 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291041 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291057 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291117 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291133 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291193 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291208 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291265 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291280 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291339 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291353 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291411 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291425 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291484 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291499 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291571 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291593 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291658 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291673 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291733 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291748 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291816 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291832 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291892 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291906 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.291972 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.291986 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292045 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292060 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292123 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292138 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292201 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292218 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292276 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292292 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292354 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292369 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292430 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292445 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292503 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292522 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292582 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292595 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292656 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292670 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292748 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292762 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292826 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292840 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292898 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292911 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.292970 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.292983 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293043 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293056 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293130 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293144 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293203 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293216 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293274 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293287 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293348 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293361 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293424 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293437 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293506 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293520 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293599 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293619 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293682 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293699 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293757 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293775 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293838 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293854 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293915 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.293931 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.293995 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294011 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294077 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294093 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294156 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294171 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294232 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294249 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294308 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294324 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294406 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294422 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294489 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294505 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294565 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294582 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294649 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294665 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294732 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294748 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294810 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294827 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294910 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.294926 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.294984 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295002 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295064 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295080 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295142 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295158 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295223 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295240 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295307 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295328 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295399 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295415 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295472 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295488 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295551 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295566 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.295624 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.295644 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W1102 00:58:11.296360 32256 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1102 00:58:11.296377 32256 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1102 00:58:11.296387 32256 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1102 00:58:11.296397 32256 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1102 00:58:11.296405 32256 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1102 00:58:11.296413 32256 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1102 00:58:11.296421 32256 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1102 00:58:11.296450 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296464 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296474 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.296483 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296493 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.296503 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296521 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.296534 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296544 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.296553 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296564 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296574 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296584 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296593 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.296603 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.296613 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.307864 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.307915 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.307993 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308011 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308075 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308091 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308151 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308167 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308263 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308280 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308342 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308358 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308418 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308436 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308501 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308517 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308577 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308592 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308652 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308669 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308729 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308743 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308808 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308823 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308884 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308898 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.308961 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.308977 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309038 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309053 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309130 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309146 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309206 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309221 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309279 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309294 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309355 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309373 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309434 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309449 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309508 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309523 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309584 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309602 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309663 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309679 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309743 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309759 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309854 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309869 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.309929 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.309945 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310010 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310026 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310085 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310100 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310182 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310199 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310275 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310291 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310355 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310374 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310442 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310457 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310521 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310537 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310598 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310613 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310674 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310690 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310749 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310764 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310826 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310842 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310904 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310920 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.310984 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.310999 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311061 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311077 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311136 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311155 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311215 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311229 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311316 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311332 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311395 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311410 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311469 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311483 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311543 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311558 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311622 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311640 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311708 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311723 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311788 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311803 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311863 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311881 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.311944 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.311961 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312025 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312042 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312129 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312146 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312208 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312226 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312288 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312303 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312368 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312384 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312444 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312458 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312518 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312533 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312593 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312608 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312669 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312684 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312745 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312759 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312820 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312837 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312900 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.312915 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.312983 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313001 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313089 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313105 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313175 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313190 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313249 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313265 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313322 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313337 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313397 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313412 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313477 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313502 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313579 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313642 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313740 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313780 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.313863 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.313901 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314042 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314094 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314179 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314218 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314312 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314350 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314453 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314491 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314594 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314632 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314717 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314755 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314849 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.314888 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.314973 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315011 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315101 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315119 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315176 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315197 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315262 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315275 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315342 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315357 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315436 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315459 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315517 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315532 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315597 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315612 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315667 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315682 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315748 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315763 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315823 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315835 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315897 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315915 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.315982 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.315998 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316060 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316075 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316132 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316147 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316213 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316228 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316294 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316310 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316374 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316389 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1102 00:58:11.316445 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1102 00:58:11.316460 32256 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.047s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.063s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.046s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.07s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.157s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.06s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.03s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.02s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.02s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.01s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.266s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I1102 00:58:17.369372 33678 files.go:57] Writing file "/tmp/TestWriteFile1500913690/001/SubDir/test1.tmp" I1102 00:58:17.369566 33678 files.go:113] Changing file mode for "/tmp/TestWriteFile1500913690/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.057s === RUN TestBootstrapChannelBuilder_BuildTasks I1102 00:58:23.656788 34755 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:23.657039 34755 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:23.657102 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:23.657151 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:23.657204 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:23.657237 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:23.657252 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:23.657291 34755 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:23.657322 34755 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:23.665187 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:23.716909 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:23.822236 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:23.878038 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:23.949736 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.024773 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.091680 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.175065 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.287299 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.798337 34755 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.19s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I1102 00:58:24.845451 34755 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:24.845604 34755 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:24.845676 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:24.845707 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:24.845741 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:24.845767 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:24.845785 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:24.845807 34755 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:24.845824 34755 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:24.845844 34755 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I1102 00:58:24.846979 34755 zones.go:93] ListHostedZonesPages { } I1102 00:58:24.923906 34755 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.08s) === RUN TestBootstrapChannelBuilder_AWSCloudController I1102 00:58:24.924181 34755 featureflag.go:160] FeatureFlag "ImageDigest"=false I1102 00:58:24.924414 34755 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:24.924499 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1102 00:58:24.924550 34755 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1102 00:58:24.924601 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1102 00:58:24.924644 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1102 00:58:24.924674 34755 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1102 00:58:24.924723 34755 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1102 00:58:24.924754 34755 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1102 00:58:24.926656 34755 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.08s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W1102 00:58:25.281088 34755 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I1102 00:58:25.313483 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313507 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313518 34755 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1102 00:58:25.313544 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1102 00:58:25.313556 34755 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I1102 00:58:25.313582 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1102 00:58:25.313601 34755 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I1102 00:58:25.313618 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313630 34755 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1102 00:58:25.313646 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313658 34755 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1102 00:58:25.313675 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313706 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313726 34755 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1102 00:58:25.313736 34755 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1102 00:58:25.313745 34755 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I1102 00:58:25.313754 34755 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I1102 00:58:25.313763 34755 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I1102 00:58:25.313776 34755 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1102 00:58:25.313786 34755 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I1102 00:58:25.313795 34755 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I1102 00:58:25.314936 34755 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.751s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I1102 00:58:21.665197 34677 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender808586032/001 I1102 00:58:21.667146 34677 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender808586032/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I1102 00:58:21.668316 34677 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender1450267557/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I1102 00:58:21.669118 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1102 00:58:21.669224 34677 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1102 00:58:21.669317 34677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I1102 00:58:21.669539 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.669635 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.669710 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.669745 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.669906 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.669945 34677 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1102 00:58:21.669988 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.670074 34677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1102 00:58:21.670181 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1102 00:58:21.670358 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.670406 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.670453 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.670475 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.670513 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.670553 34677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1102 00:58:21.670618 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I1102 00:58:21.670941 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.670995 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.671102 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.671145 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.671176 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.671205 34677 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1102 00:58:21.671327 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.671536 34677 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1102 00:58:21.671596 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.671641 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.671674 34677 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1102 00:58:21.671748 34677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.671935 34677 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I1102 00:58:21.672012 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.672042 34677 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I1102 00:58:21.672092 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1102 00:58:21.672250 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.672300 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.672353 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.672374 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.672404 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.672433 34677 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1102 00:58:21.672478 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.672543 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.672575 34677 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1102 00:58:21.672621 34677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.672675 34677 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I1102 00:58:21.672712 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1102 00:58:21.672760 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I1102 00:58:21.672924 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1102 00:58:21.672977 34677 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1102 00:58:21.673006 34677 internetgateways.go:66] CreateInternetGateway: { } I1102 00:58:21.673036 34677 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I1102 00:58:21.673168 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.673216 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.673260 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.673280 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.673357 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.673387 34677 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1102 00:58:21.673424 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.673492 34677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1102 00:58:21.673611 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1102 00:58:21.673743 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.673790 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.673836 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.673855 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.673894 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.673938 34677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1102 00:58:21.674004 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I1102 00:58:21.674915 34677 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2180142906/001 I1102 00:58:21.675177 34677 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2180142906/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I1102 00:58:21.677918 34677 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1458021201/001 I1102 00:58:21.679833 34677 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1458021201/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I1102 00:58:21.680502 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.680569 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.680695 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.680750 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.680785 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.680817 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.680885 34677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1102 00:58:21.681097 34677 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I1102 00:58:21.681148 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1102 00:58:21.681186 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1102 00:58:21.681332 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.681384 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.681440 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.681461 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.681493 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.681522 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.681569 34677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1102 00:58:21.681655 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I1102 00:58:21.682099 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.682157 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.682254 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.682298 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.682326 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.682354 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.682402 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.682523 34677 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1102 00:58:21.682573 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.682606 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.682649 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1102 00:58:21.682790 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.682840 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.682904 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.682925 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.682957 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.682985 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.683031 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.683098 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.683130 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I1102 00:58:21.683423 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.683472 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.683629 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.683674 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.683701 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.683730 34677 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1102 00:58:21.683781 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.683907 34677 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1102 00:58:21.683954 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.684141 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.684179 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.684205 34677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:58:21.684264 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1102 00:58:21.684428 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.684481 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.684534 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.684559 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.684593 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.684621 34677 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1102 00:58:21.684682 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.684748 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.684818 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.684846 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I1102 00:58:21.685225 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.685270 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.685354 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.685396 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.685419 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.685442 34677 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1102 00:58:21.685481 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.685528 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.685649 34677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1102 00:58:21.685660 34677 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1102 00:58:21.685708 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.685730 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.685753 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1102 00:58:21.685918 34677 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1102 00:58:21.685960 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.686002 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.686017 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.686040 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.686062 34677 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1102 00:58:21.686113 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.686170 34677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1102 00:58:21.686261 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.686287 34677 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I1102 00:58:21.686429 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I1102 00:58:21.686491 34677 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I1102 00:58:21.686635 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.686675 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.686712 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.686730 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.686791 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.686815 34677 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1102 00:58:21.686849 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.686901 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.687014 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1102 00:58:21.687037 34677 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1102 00:58:21.687072 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1102 00:58:21.687234 34677 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1102 00:58:21.687273 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.687309 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.687324 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.687353 34677 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1102 00:58:21.687385 34677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1102 00:58:21.687439 34677 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I1102 00:58:21.687664 34677 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.687707 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.687786 34677 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1102 00:58:21.687820 34677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1102 00:58:21.687840 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.687861 34677 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.687966 34677 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.688005 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1102 00:58:21.688044 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.688059 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.688080 34677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1102 00:58:21.688101 34677 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I1102 00:58:21.688268 34677 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I1102 00:58:21.688389 34677 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.688429 34677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I1102 00:58:21.688471 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1102 00:58:21.688486 34677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1102 00:58:21.688515 34677 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.069s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.042s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.054s === RUN TestDiskRenderAzure I1102 00:58:18.324411 34113 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I1102 00:58:18.324801 34113 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I1102 00:58:18.325158 34113 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I1102 00:58:18.325880 34113 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I1102 00:58:18.326169 34113 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I1102 00:58:18.326489 34113 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I1102 00:58:18.327123 34113 resourcegroup.go:103] Creating a new Resource Group with name: rg I1102 00:58:18.327176 34113 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I1102 00:58:18.327634 34113 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I1102 00:58:18.329091 34113 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I1102 00:58:18.329794 34113 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I1102 00:58:18.330161 34113 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I1102 00:58:18.330544 34113 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I1102 00:58:18.330895 34113 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.048s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.079s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I1102 00:58:21.053756 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.054514 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.054712 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.055265 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.055459 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.055673 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I1102 00:58:21.055993 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.056130 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.056279 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.056450 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.056611 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.056691 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I1102 00:58:21.056992 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.057313 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.057466 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.057910 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1102 00:58:21.058070 34648 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1102 00:58:21.058245 34648 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.063s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.053s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.029s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.071s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.035s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.069s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I1102 00:58:23.989047 34788 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I1102 00:58:23.989102 34788 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I1102 00:58:23.989115 34788 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I1102 00:58:23.989126 34788 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I1102 00:58:23.989138 34788 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I1102 00:58:23.989148 34788 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I1102 00:58:23.989158 34788 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W1102 00:58:23.989537 34788 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.059s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.023s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.054s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.036s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.052s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.009s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W1102 00:58:19.658566 34451 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.018s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.012s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.024s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.025s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.020s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W1102 00:58:24.369831 34862 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.066s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 129.0 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.1-r1.apk >>> kops: Build complete at Wed, 02 Nov 2022 00:59:26 +0000 elapsed time 0h 3m 28s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20221102.005559) (2/3) Purging kubernetes (1.25.0-r3) (3/3) Purging go (1.19.3-r0) Executing busybox-1.35.0-r27.trigger OK: 393 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...