>>> kops: Building testing/kops 1.24.1-r1 (using abuild 3.9.0-r6) started Thu, 08 Sep 2022 14:14:28 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/2) Installing kubernetes (1.25.0-r1) (2/2) Installing .makedepends-kops (20220908.141429) OK: 807 MiB in 106 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 76 29.4M 76 22.6M 0 0 34.8M 0 --:--:-- --:--:-- --:--:-- 34.8M 100 29.4M 100 29.4M 0 0 41.6M 0 --:--:-- --:--:-- --:--:-- 41.5M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.1.tar.gz >>> kops: Checking sha512sums... kops-1.24.1.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.24.1.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.60 go: downloading google.golang.org/api v0.88.0 go: downloading k8s.io/apimachinery v0.24.3 go: downloading k8s.io/klog/v2 v2.70.1 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.8.2 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading k8s.io/api v0.24.3 go: downloading github.com/spf13/cobra v1.5.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/cli-runtime v0.24.3 go: downloading k8s.io/client-go v0.24.3 go: downloading golang.org/x/oauth2 v0.0.0-20220718184931-c8730f7fcb92 go: downloading golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d go: downloading github.com/stretchr/testify v1.8.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading cloud.google.com/go/compute v1.7.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.24.3 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading k8s.io/utils v0.0.0-20220713171938-56c0de1e6f5e go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.3.3 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading helm.sh/helm/v3 v3.9.2 go: downloading k8s.io/kubectl v0.24.3 go: downloading sigs.k8s.io/controller-runtime v0.12.3 go: downloading github.com/prometheus/client_golang v1.12.2 go: downloading k8s.io/component-base v0.24.3 go: downloading github.com/google/go-cmp v0.5.8 go: downloading github.com/digitalocean/godo v1.81.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading golang.org/x/net v0.0.0-20220708220712-1185a9018129 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.5.7-v3refs go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.66.6 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.1 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading github.com/hetznercloud/hcloud-go v1.35.1 go: downloading k8s.io/kubelet v0.24.3 go: downloading k8s.io/mount-utils v0.24.3 go: downloading golang.org/x/sys v0.0.0-20220721230656-c6bc011c0c49 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.27 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/spotinst/spotinst-sdk-go v1.124.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f go: downloading github.com/google/go-tpm-tools v0.3.8 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.10.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading golang.org/x/text v0.3.7 go: downloading sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.11.4 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.6 go: downloading k8s.io/kube-openapi v0.0.0-20220328201542-3ee0da9b0b42 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.0 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading k8s.io/apiextensions-apiserver v0.24.2 go: downloading sigs.k8s.io/gateway-api v0.4.1 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.24.3 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.32.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.18 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading google.golang.org/grpc v1.47.0 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading k8s.io/csi-translation-lib v0.24.3 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.7.3 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading golang.org/x/tools v0.1.8 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.18.1 go: downloading go.uber.org/goleak v1.1.12 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.2 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/emicklei/go-restful v2.9.5+incompatible go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/josharian/intern v1.0.0 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/moby/sys/mountinfo v0.5.0 go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.24.1/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.24.1/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220809-2728-gacb34b734c" k8s.io/kops/cmd/kops go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0908 14:17:08.048803 23887 addon.go:229] addon test wants to update all nodes I0908 14:17:08.051273 23887 addon.go:229] addon test wants to update worker nodes I0908 14:17:08.051727 23887 addon.go:229] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0908 14:17:08.052231 23887 addon.go:262] installing PKI for "test" I0908 14:17:08.426136 23887 addon.go:262] installing PKI for "test" --- PASS: Test_InstallPKI (0.58s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 0.631s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.041s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0908 14:17:37.498697 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.499029 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.499085 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.499127 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.499177 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.499213 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.499233 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.499269 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.499294 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.499364 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.499398 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.572201 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:37.572271 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:37.572647 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:37.599281 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.599457 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.599494 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.599533 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.599566 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.599591 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.599607 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.599628 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.599644 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.599704 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.599730 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.745659 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:37.745756 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:37.746074 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:37.752379 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.752541 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.752579 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.752607 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.752639 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.752664 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.752681 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.752708 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.752725 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.752790 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.752816 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.797639 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:37.797731 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:37.798053 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:37.804054 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.804205 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.804237 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.804268 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.804299 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.804325 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.804344 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.804370 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.804387 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.804462 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.804489 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.878042 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:37.878144 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:37.878547 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:37.888308 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.888519 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.888563 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.888602 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.888644 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.888680 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.888706 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.888737 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.888760 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.888832 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:37.888867 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.018731 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.018809 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.019130 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.026882 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.027117 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.027167 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.027213 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.027260 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.027319 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.027347 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.027386 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.027411 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.027500 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.027539 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.138838 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.138919 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.139204 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.65s) === RUN TestCreateClusterOverride I0908 14:17:38.145126 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.145304 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.145337 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.145365 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.145394 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.145467 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.145484 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.145505 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.145521 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.145605 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.145633 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.202146 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.202550 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.202819 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterComplex I0908 14:17:38.208658 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.208807 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.208839 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.208868 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.208901 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.208953 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.208971 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.208996 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.209013 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.209071 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.209096 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.258806 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.258953 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.259526 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.06s) === RUN TestCreateClusterHA I0908 14:17:38.272327 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.272607 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.272662 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.272717 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.272770 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.272861 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.272893 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.272933 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.272960 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.273058 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.273101 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.369431 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.369490 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.369503 29538 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0908 14:17:38.369513 29538 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0908 14:17:38.369768 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.475898 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.476131 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.476194 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.476248 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.476297 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.476338 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.476368 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.476404 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.476431 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.476523 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.476564 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.560068 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.560169 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.560193 29538 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0908 14:17:38.560210 29538 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0908 14:17:38.560668 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.31s) === RUN TestCreateClusterHAGCE I0908 14:17:38.585865 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.586099 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.586147 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.586194 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.586239 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.586322 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.586352 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.586386 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.586410 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.586483 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.586522 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.629132 29538 new_cluster.go:1168] Cloud Provider ID = gce I0908 14:17:38.629230 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.09s) === RUN TestCreateClusterGCE I0908 14:17:38.677901 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.678143 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.678204 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.678253 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.678299 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.678375 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.678404 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.678443 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.678468 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.678547 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.678598 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.816486 29538 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0908 14:17:38.816543 29538 new_cluster.go:1168] Cloud Provider ID = gce I0908 14:17:38.816584 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.16s) === RUN TestCreateClusterHASharedZone I0908 14:17:38.837850 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.837997 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.838037 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.838065 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.838095 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.838156 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.838175 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.838207 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.838224 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.838285 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.838313 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0908 14:17:38.884833 29538 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0908 14:17:38.884897 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.884963 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.885263 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.06s) === RUN TestCreateClusterHASharedZones I0908 14:17:38.893448 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.893618 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.893653 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.893682 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.893712 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.893769 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.893785 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.893807 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.893823 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.893885 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:38.893912 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0908 14:17:38.989196 29538 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0908 14:17:38.989264 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:38.989331 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:38.989346 29538 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0908 14:17:38.989662 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.11s) === RUN TestCreateClusterPrivate I0908 14:17:39.003527 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.003701 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.003736 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.003764 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.003794 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.003853 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.003873 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.003896 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.003913 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.003990 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.004029 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.031421 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.031555 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:39.031582 29538 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0908 14:17:39.032074 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:39.032128 29538 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.04s) === RUN TestCreateClusterPrivateGCE I0908 14:17:39.045211 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.045447 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.045491 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.045538 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.045584 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.045658 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.045690 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.045727 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.045752 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.045834 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.045876 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.173199 29538 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0908 14:17:39.173264 29538 new_cluster.go:1168] Cloud Provider ID = gce I0908 14:17:39.173311 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.17s) === RUN TestCreateClusterWithNGWSpecified I0908 14:17:39.217146 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.217371 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.217410 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.217452 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.217491 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.217555 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.217584 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.217615 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.217635 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.217702 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.217739 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.317018 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.317112 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:39.317139 29538 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0908 14:17:39.317562 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:39.317611 29538 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.11s) === RUN TestCreateClusterWithINGWSpecified I0908 14:17:39.328930 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.329152 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.329195 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.329235 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.329284 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.329361 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.329391 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.329426 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.329450 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.329527 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.329567 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.424568 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.424663 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:39.424690 29538 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0908 14:17:39.425129 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:39.425180 29538 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.11s) === RUN TestCreateClusterSharedVPC I0908 14:17:39.436514 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.436736 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.436778 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.436825 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.436870 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.436946 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.436977 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.437012 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.437036 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.437113 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.437154 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.490334 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.490395 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.490439 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.490479 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.490498 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.490542 29538 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0908 14:17:39.490800 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.11s) === RUN TestCreateClusterSharedSubnets I0908 14:17:39.545955 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.546148 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.546189 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.546227 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.546267 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.546329 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.546357 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.546385 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.546405 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.546504 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.546542 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.639664 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.639718 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.639818 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.639877 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.639913 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.639953 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.639980 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.640352 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.13s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0908 14:17:39.671551 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.671748 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.671783 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.671817 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.671853 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.671914 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.671937 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.671963 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.671982 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.672041 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.672075 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.799586 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:39.799635 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.799658 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.799728 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.799773 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.799790 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.799815 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.799830 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.800135 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.17s) === RUN TestCreateClusterPrivateSharedSubnets I0908 14:17:39.837210 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.837371 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.837403 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.837430 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.837463 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.837515 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.837533 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.837556 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.837573 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.837630 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.837655 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.921342 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.921387 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.921462 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.921482 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.921510 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:39.921549 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.921566 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.921588 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:17:39.921601 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0908 14:17:39.921921 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.12s) === RUN TestCreateClusterIPv6 I0908 14:17:39.959957 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:39.960123 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:39.960156 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:39.960228 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:39.960258 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:39.960283 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:39.960300 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:39.960324 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:39.960340 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:39.960396 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:39.960423 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:40.159087 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:40.159169 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:40.159578 29538 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.24s) === RUN TestCreateClusterKarpenter I0908 14:17:40.204844 29538 featureflag.go:162] FeatureFlag "Karpenter"=true I0908 14:17:40.204861 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.205012 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.205046 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.205077 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.205108 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.205162 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.205180 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.205203 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.205219 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.205277 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:40.205301 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0908 14:17:40.303523 29538 new_cluster.go:1168] Cloud Provider ID = aws I0908 14:17:40.303602 29538 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0908 14:17:40.303617 29538 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0908 14:17:40.303627 29538 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0908 14:17:40.303983 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:40.325118 29538 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.12s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0908 14:17:40.325356 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.325494 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.325531 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.325557 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.325587 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.325635 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.325653 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.325676 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.325694 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.386077 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:40.388980 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:40.389120 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:17:43.371329 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:46.561295 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:17:46.561810 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:46.563146 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:46.563172 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:46.563244 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:46.563786 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:46.563913 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:46.564913 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:17:46.572981 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:17:46.573887 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.574637 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.575356 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.576055 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.576135 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:17:46.576487 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:17:46.632394 29538 target.go:112] Terraform output is in /tmp/test657244905/out I0908 14:17:46.677206 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:46.677357 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:46.677394 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:46.677424 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:46.677458 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:46.677484 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:46.677501 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:46.677526 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:46.677544 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:46.691094 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:46.692686 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.692743 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:17:46.733454 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:46.806268 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:17:46.806718 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:17:46.806875 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:46.806892 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:17:46.806918 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:46.806974 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:17:46.807371 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:46.807625 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:17:46.807651 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:46.807812 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:17:46.809095 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:17:46.817711 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:17:46.818022 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.818259 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.818731 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.820908 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.821219 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:17:46.821491 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:17:46.826479 29538 target.go:168] Cloudformation output is in /tmp/test690924797/out --- PASS: TestMinimal (6.52s) === RUN TestMinimal_v1_23 I0908 14:17:46.845448 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:46.845689 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:46.845744 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:46.845790 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:46.845843 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:46.845924 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:46.845956 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:46.845998 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:46.846028 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:46.884163 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0908 14:17:46.887689 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:46.887789 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:47.952527 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:48.116611 29538 executor.go:111] Tasks: 0 done / 90 total; 45 can run W0908 14:17:48.117204 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:48.117251 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:48.118831 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:48.119180 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:48.119249 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:48.119383 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:48.124218 29538 executor.go:111] Tasks: 45 done / 90 total; 19 can run I0908 14:17:48.132581 29538 executor.go:111] Tasks: 64 done / 90 total; 24 can run I0908 14:17:48.132866 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.133245 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.134146 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.136256 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.136379 29538 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0908 14:17:48.136686 29538 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0908 14:17:48.196774 29538 target.go:112] Terraform output is in /tmp/test1881484791/out I0908 14:17:48.241138 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:48.241286 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:48.241320 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:48.241351 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:48.241382 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:48.241407 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:48.241423 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:48.241447 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:48.241464 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:48.254540 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:48.256078 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.256134 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:17:48.296960 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:48.395105 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:17:48.395506 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:48.395639 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:17:48.395658 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:48.395741 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:17:48.395768 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:17:48.395790 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:17:48.395916 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:48.395939 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:17:48.396063 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:48.398667 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:17:48.407892 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:17:48.408264 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.408491 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.409451 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.410852 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.410997 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:17:48.411288 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:17:48.414429 29538 target.go:168] Cloudformation output is in /tmp/test3485794100/out --- PASS: TestMinimal_v1_23 (1.58s) === RUN TestMinimal_v1_24 I0908 14:17:48.424657 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:48.424890 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:48.424943 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:48.424987 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:48.425037 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:48.425117 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:48.425145 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:48.425186 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:48.425211 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:48.454432 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:48.457532 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:48.457633 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:49.940171 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:50.115729 29538 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0908 14:17:50.115988 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:50.116287 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:50.116792 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:50.116778 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:50.116845 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:50.117066 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:50.119463 29538 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0908 14:17:50.128769 29538 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0908 14:17:50.129205 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.129413 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.130602 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.131788 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.131953 29538 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0908 14:17:50.132282 29538 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0908 14:17:50.187182 29538 target.go:112] Terraform output is in /tmp/test1057578280/out I0908 14:17:50.208690 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:50.208820 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:50.208853 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:50.208880 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:50.208912 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:50.208937 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:50.208961 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:50.208983 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:50.209000 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:50.222197 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:50.223732 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.223784 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:17:50.264079 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:50.362129 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:17:50.362866 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:50.362960 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:17:50.362998 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:50.362996 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:17:50.363128 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:17:50.363147 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:50.363127 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:17:50.363232 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:50.363237 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:17:50.365052 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:17:50.370074 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:17:50.370383 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.370556 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.373305 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.376008 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.376173 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:17:50.376425 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:17:50.379287 29538 target.go:168] Cloudformation output is in /tmp/test2193231002/out --- PASS: TestMinimal_v1_24 (1.96s) === RUN TestNvidia I0908 14:17:50.389344 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:50.389597 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:50.389646 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:50.389692 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:50.389738 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:50.389816 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:50.389845 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:50.389884 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:50.389910 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:50.460023 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:50.461670 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.461729 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.500809 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:50.607750 29538 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0908 14:17:50.608340 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:50.608436 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:50.608474 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:50.608601 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:50.608621 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:50.608792 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:50.610512 29538 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0908 14:17:50.619854 29538 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0908 14:17:50.620249 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.620506 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.621847 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.623002 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:50.623141 29538 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0908 14:17:50.623684 29538 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0908 14:17:50.680823 29538 target.go:112] Terraform output is in /tmp/test3598663091/out I0908 14:17:51.037208 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:51.037465 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:51.037520 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:51.037568 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:51.037620 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:51.037663 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:51.037693 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:51.037731 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:51.037758 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:51.062177 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:51.065115 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.065210 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.104862 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:51.210469 29538 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0908 14:17:51.211196 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:17:51.211243 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:51.211359 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:17:51.211179 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:51.211402 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:17:51.211486 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:17:51.211513 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:17:51.211561 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:51.211779 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:17:51.215771 29538 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0908 14:17:51.222748 29538 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0908 14:17:51.223145 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.223449 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.224565 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.225595 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.225737 29538 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0908 14:17:51.226046 29538 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0908 14:17:51.229333 29538 target.go:168] Cloudformation output is in /tmp/test390209173/out --- PASS: TestNvidia (0.90s) === RUN TestMinimalGossip I0908 14:17:51.289570 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:51.289808 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:51.289859 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:51.289908 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:51.289960 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:51.290044 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:51.290077 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:51.290117 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:51.290145 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:51.368313 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.368414 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.408584 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0908 14:17:51.514094 29538 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0908 14:17:51.514633 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:51.514958 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:51.515125 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:51.517458 29538 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0908 14:17:51.522651 29538 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0908 14:17:51.522850 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.523194 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.525851 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.529547 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.529682 29538 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0908 14:17:51.530047 29538 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0908 14:17:51.579179 29538 target.go:112] Terraform output is in /tmp/test1257772426/out --- PASS: TestMinimalGossip (0.50s) === RUN TestMinimalGossipIRSA I0908 14:17:51.793776 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:51.794069 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:51.794125 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:51.794178 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:51.794234 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:51.794326 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:51.794355 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:51.794393 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:51.794423 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:51.825681 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.825786 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.866548 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0908 14:17:51.979184 29538 executor.go:111] Tasks: 0 done / 87 total; 44 can run W0908 14:17:51.979660 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:51.979871 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:51.980053 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:51.985725 29538 executor.go:111] Tasks: 44 done / 87 total; 20 can run I0908 14:17:51.991159 29538 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0908 14:17:51.991521 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.991771 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.993336 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.994184 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:51.994305 29538 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0908 14:17:51.994620 29538 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0908 14:17:52.049478 29538 target.go:112] Terraform output is in /tmp/test1225352172/out --- PASS: TestMinimalGossipIRSA (0.35s) === RUN TestMinimalGCE I0908 14:17:52.142852 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:52.147520 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0908 14:17:53.117437 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:53.117514 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:53.117618 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0908 14:17:53.117650 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0908 14:17:53.117670 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:17:53.117695 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:17:53.117764 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:17:53.121987 29538 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0908 14:17:53.125303 29538 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0908 14:17:53.130905 29538 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0908 14:17:53.133688 29538 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0908 14:17:53.133971 29538 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0908 14:17:53.153953 29538 target.go:112] Terraform output is in /tmp/test1325197496/out --- PASS: TestMinimalGCE (1.05s) === RUN TestMinimalGCEPrivate I0908 14:17:53.197607 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:53.203377 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0908 14:17:54.086691 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:54.086786 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:54.086924 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0908 14:17:54.086961 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0908 14:17:54.086987 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0908 14:17:54.087025 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0908 14:17:54.087100 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:17:54.088974 29538 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0908 14:17:54.092283 29538 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0908 14:17:54.098262 29538 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0908 14:17:54.100966 29538 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0908 14:17:54.102360 29538 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0908 14:17:54.102544 29538 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0908 14:17:54.134455 29538 target.go:112] Terraform output is in /tmp/test1766038431/out --- PASS: TestMinimalGCEPrivate (0.97s) === RUN TestMinimalGCEInternalLoadBalancer I0908 14:17:54.164768 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:54.166020 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0908 14:17:54.341281 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:54.341355 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:54.341497 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0908 14:17:54.341532 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0908 14:17:54.341558 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0908 14:17:54.341595 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0908 14:17:54.341666 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0908 14:17:54.341716 29538 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0908 14:17:54.343461 29538 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0908 14:17:54.346205 29538 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0908 14:17:54.351411 29538 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0908 14:17:54.351463 29538 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0908 14:17:54.354332 29538 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0908 14:17:54.355600 29538 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0908 14:17:54.356016 29538 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0908 14:17:54.356299 29538 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0908 14:17:54.388551 29538 target.go:112] Terraform output is in /tmp/test931646468/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.24s) === RUN TestMinimalGCELongClusterName I0908 14:17:54.408690 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:54.413261 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0908 14:17:54.639317 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:54.639420 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:54.639554 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0908 14:17:54.639591 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0908 14:17:54.639618 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0908 14:17:54.639653 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0908 14:17:54.639727 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:17:54.641383 29538 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0908 14:17:54.648775 29538 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0908 14:17:54.654375 29538 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0908 14:17:54.657194 29538 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0908 14:17:54.657545 29538 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0908 14:17:54.689899 29538 target.go:112] Terraform output is in /tmp/test3158426949/out --- PASS: TestMinimalGCELongClusterName (0.30s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0908 14:17:54.712857 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:54.714227 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0908 14:17:54.952055 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:54.952165 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:54.952313 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0908 14:17:54.952354 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0908 14:17:54.952382 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0908 14:17:54.952418 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0908 14:17:54.952490 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0908 14:17:54.952545 29538 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0908 14:17:54.954380 29538 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0908 14:17:54.957095 29538 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0908 14:17:54.962563 29538 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0908 14:17:54.962605 29538 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0908 14:17:54.965228 29538 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0908 14:17:54.965440 29538 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0908 14:17:54.965636 29538 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0908 14:17:54.965785 29538 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0908 14:17:55.002575 29538 target.go:112] Terraform output is in /tmp/test2984195217/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.30s) === RUN TestHA I0908 14:17:55.015184 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:55.015436 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:55.015498 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:55.015551 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:55.015605 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:55.015719 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:55.015752 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:55.015792 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:55.015821 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:55.066018 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:55.069182 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.069301 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.069365 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.069425 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.110916 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:55.213625 29538 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0908 14:17:55.214086 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:55.214206 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:55.214232 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:55.214264 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:55.214372 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:55.214444 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:55.214601 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:55.214634 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:55.216638 29538 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0908 14:17:55.228679 29538 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0908 14:17:55.229216 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.229467 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.229745 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.229998 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.231979 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.232129 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.232235 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.233421 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.233538 29538 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0908 14:17:55.233966 29538 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0908 14:17:55.312465 29538 target.go:112] Terraform output is in /tmp/test1406401155/out --- PASS: TestHA (0.32s) === RUN TestHighAvailabilityGCE I0908 14:17:55.334194 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:55.335594 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0908 14:17:55.516829 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:17:55.516926 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:17:55.517045 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0908 14:17:55.517103 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0908 14:17:55.517130 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0908 14:17:55.517153 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0908 14:17:55.517221 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0908 14:17:55.517321 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0908 14:17:55.517388 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:17:55.519539 29538 executor.go:111] Tasks: 0 done / 79 total; 44 can run I0908 14:17:55.522432 29538 executor.go:111] Tasks: 44 done / 79 total; 20 can run I0908 14:17:55.527660 29538 executor.go:111] Tasks: 64 done / 79 total; 7 can run I0908 14:17:55.536600 29538 executor.go:111] Tasks: 71 done / 79 total; 5 can run I0908 14:17:55.537932 29538 executor.go:111] Tasks: 76 done / 79 total; 3 can run I0908 14:17:55.538202 29538 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0908 14:17:55.578590 29538 target.go:112] Terraform output is in /tmp/test3388174741/out --- PASS: TestHighAvailabilityGCE (0.37s) === RUN TestComplex I0908 14:17:55.703493 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:55.703643 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:55.703679 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:55.703744 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:55.703780 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:55.703808 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:55.703826 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:55.703852 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:55.703870 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:55.732442 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:55.734780 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:55.734845 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:56.702471 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:56.894449 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run W0908 14:17:56.894893 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:56.895281 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:56.895421 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:56.895465 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:56.895592 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:56.897912 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:17:56.904812 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:17:56.905845 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:56.906418 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:56.910767 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:56.912629 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:56.912782 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:17:56.913252 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0908 14:17:56.988428 29538 target.go:112] Terraform output is in /tmp/test2262541418/out I0908 14:17:57.007836 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:57.007974 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:57.008016 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:57.008049 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:57.008088 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:57.008119 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:57.008137 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:57.008161 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:57.008179 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:57.039289 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:57.043440 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.043547 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.085118 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:57.280834 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0908 14:17:57.281340 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:17:57.281383 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:57.281522 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:17:57.281549 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:17:57.281970 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:57.285758 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:17:57.294792 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:17:57.295548 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.295773 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.296932 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.298747 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.298898 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:17:57.299253 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0908 14:17:57.303856 29538 target.go:168] Cloudformation output is in /tmp/test2378255158/out I0908 14:17:57.330380 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:57.330595 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:57.330683 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:57.330734 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:57.330789 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:57.330833 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:57.330861 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:57.330898 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:57.330923 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:57.368120 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:57.372201 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.372307 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.412717 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:57.609421 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run W0908 14:17:57.610025 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:57.610352 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:57.610525 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:57.610566 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:57.611674 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:57.617074 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:17:57.622917 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:17:57.623613 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.623901 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.624920 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.626629 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.626750 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:17:57.627078 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0908 14:17:57.700819 29538 target.go:112] Terraform output is in /tmp/test971918776/out --- PASS: TestComplex (2.00s) === RUN TestCompress I0908 14:17:57.705127 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:57.705362 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:57.705419 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:57.705470 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:57.705522 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:57.705610 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:57.705639 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:57.705678 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:57.705708 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:57.732226 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:57.735163 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.735264 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.775341 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:57.871412 29538 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0908 14:17:57.872439 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:57.872487 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:57.872742 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:57.873350 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:57.873397 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:57.876996 29538 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0908 14:17:57.883142 29538 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0908 14:17:57.883514 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.883739 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.893903 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.894984 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.895119 29538 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0908 14:17:57.895506 29538 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0908 14:17:57.947908 29538 target.go:112] Terraform output is in /tmp/test359164870/out --- PASS: TestCompress (0.26s) === RUN TestExternalPolicies I0908 14:17:57.961791 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:57.961948 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:57.961984 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:57.962014 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:57.962047 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:57.962104 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:57.962122 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:57.962148 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:57.962165 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:57.978556 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:57.980502 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:57.980557 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.020171 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:58.120208 29538 executor.go:111] Tasks: 0 done / 98 total; 45 can run I0908 14:17:58.120486 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:58.120514 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:58.120633 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:58.120671 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:58.120825 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:58.120972 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:58.127759 29538 executor.go:111] Tasks: 45 done / 98 total; 20 can run I0908 14:17:58.133135 29538 executor.go:111] Tasks: 65 done / 98 total; 30 can run I0908 14:17:58.133396 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.133586 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.134688 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.136197 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.136318 29538 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0908 14:17:58.136644 29538 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0908 14:17:58.204277 29538 target.go:112] Terraform output is in /tmp/test857086349/out --- PASS: TestExternalPolicies (0.28s) === RUN TestMinimalIPv6 I0908 14:17:58.240477 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:58.240626 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:58.240663 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:58.240693 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:58.240726 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:58.240778 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:58.240795 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:58.240817 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:58.240835 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:58.255263 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:58.257101 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.257160 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.297187 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:58.469486 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0908 14:17:58.470157 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:17:58.470193 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:58.470457 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:58.470539 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:58.470657 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:58.470759 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:58.472772 29538 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0908 14:17:58.482011 29538 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0908 14:17:58.482341 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.482738 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.483816 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.484940 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.485066 29538 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0908 14:17:58.485782 29538 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0908 14:17:58.486073 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:17:58.551884 29538 target.go:112] Terraform output is in /tmp/test1236038428/out I0908 14:17:58.572492 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:58.572628 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:58.572665 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:58.572698 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:58.572731 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:58.572759 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:58.572775 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:58.572807 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:58.572825 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:58.587398 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:17:58.589301 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.589363 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.629581 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:17:58.803845 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:17:58.804190 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0908 14:17:58.804315 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:58.804230 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:58.804370 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:17:58.806511 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:17:58.806615 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:58.806853 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:17:58.806884 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:58.807098 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:17:58.811403 29538 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0908 14:17:58.816689 29538 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0908 14:17:58.817267 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.817685 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.818741 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.820476 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.820749 29538 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0908 14:17:58.821246 29538 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0908 14:17:58.821538 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:17:58.825651 29538 target.go:168] Cloudformation output is in /tmp/test2234812397/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I0908 14:17:58.844055 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:58.844303 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:58.844359 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:58.844407 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:58.844461 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:58.844549 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:58.844579 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:58.844617 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:58.844646 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:58.873659 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0908 14:17:58.877325 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.877422 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:58.917532 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:59.098788 29538 executor.go:111] Tasks: 0 done / 122 total; 46 can run W0908 14:17:59.099055 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:59.099384 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:17:59.099648 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:59.099709 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:59.099800 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:17:59.099919 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:59.101987 29538 executor.go:111] Tasks: 46 done / 122 total; 23 can run I0908 14:17:59.111950 29538 executor.go:111] Tasks: 69 done / 122 total; 36 can run I0908 14:17:59.112538 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.112797 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.113964 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.115172 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.115298 29538 executor.go:111] Tasks: 105 done / 122 total; 11 can run I0908 14:17:59.115797 29538 executor.go:111] Tasks: 116 done / 122 total; 6 can run I0908 14:17:59.116163 29538 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0908 14:17:59.199141 29538 target.go:112] Terraform output is in /tmp/test3480474577/out --- PASS: TestMinimalIPv6Private (0.37s) === RUN TestMinimalIPv6Calico I0908 14:17:59.209645 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:59.209790 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:59.209826 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:59.209889 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:59.209923 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:59.209952 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:59.209970 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:59.209992 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:59.210010 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:59.237908 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:59.239772 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.239831 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.281743 29538 zones.go:93] ListHostedZonesPages { } I0908 14:17:59.700229 29538 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0908 14:17:59.700457 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:59.701071 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:17:59.702516 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:17:59.704356 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:17:59.701107 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:17:59.705989 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:17:59.707760 29538 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0908 14:17:59.713471 29538 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0908 14:17:59.713696 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.714092 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.714989 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.720187 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.720346 29538 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0908 14:17:59.720801 29538 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0908 14:17:59.721120 29538 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0908 14:17:59.787643 29538 target.go:112] Terraform output is in /tmp/test3820128319/out I0908 14:17:59.805153 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:59.805301 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:59.805338 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:59.805368 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:59.805401 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:59.805427 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:59.805444 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:59.805468 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:59.805485 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:59.822090 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:17:59.824050 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.824112 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:17:59.862459 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:00.287404 29538 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0908 14:18:00.287707 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:00.289145 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:00.291382 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:00.293542 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:00.293595 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:00.293751 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:00.293779 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:00.293870 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:00.293771 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:00.296140 29538 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0908 14:18:00.302424 29538 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0908 14:18:00.302669 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.302991 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.303904 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.305716 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.305884 29538 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0908 14:18:00.306266 29538 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0908 14:18:00.306565 29538 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0908 14:18:00.310252 29538 target.go:168] Cloudformation output is in /tmp/test1177226014/out --- PASS: TestMinimalIPv6Calico (1.12s) === RUN TestMinimalIPv6Cilium I0908 14:18:00.327134 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:00.327422 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:00.327483 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:00.327537 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:00.327604 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:00.327783 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:00.327877 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:00.327985 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:00.328057 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:00.355169 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:00.358689 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.358786 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.399605 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:00.619202 29538 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0908 14:18:00.619749 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:00.619987 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:00.620020 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:00.620123 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:00.620132 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:00.621874 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:00.626989 29538 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0908 14:18:00.632987 29538 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0908 14:18:00.633460 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.633707 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.634788 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.636823 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.636970 29538 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0908 14:18:00.637767 29538 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0908 14:18:00.638134 29538 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0908 14:18:00.705975 29538 target.go:112] Terraform output is in /tmp/test588493890/out I0908 14:18:00.712895 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:00.713034 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:00.713075 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:00.713106 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:00.713138 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:00.713166 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:00.713183 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:00.713210 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:00.713226 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:00.727772 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:00.729788 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.729849 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.769271 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:00.983485 29538 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0908 14:18:00.984246 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:00.984277 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:00.984400 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:00.985433 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:00.984459 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:00.985872 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:00.985899 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:00.986134 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } W0908 14:18:00.986372 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:00.992377 29538 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0908 14:18:00.997978 29538 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0908 14:18:00.998645 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:00.998943 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.000526 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.002464 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.002634 29538 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0908 14:18:01.003074 29538 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0908 14:18:01.003380 29538 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0908 14:18:01.007082 29538 target.go:168] Cloudformation output is in /tmp/test4024884986/out --- PASS: TestMinimalIPv6Cilium (0.71s) === RUN TestMinimalWarmPool I0908 14:18:01.038485 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:01.038725 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:01.038777 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:01.038826 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:01.038890 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:01.038973 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:01.039002 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:01.039041 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:01.039071 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:01.069340 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:01.072677 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.072780 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.112714 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:01.316115 29538 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0908 14:18:01.316836 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:01.317190 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:01.318673 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:01.318772 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:01.318984 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:01.319361 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:01.323893 29538 executor.go:111] Tasks: 45 done / 88 total; 19 can run I0908 14:18:01.329592 29538 executor.go:111] Tasks: 64 done / 88 total; 21 can run I0908 14:18:01.330046 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.330389 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.331339 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.333021 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.333144 29538 executor.go:111] Tasks: 85 done / 88 total; 2 can run I0908 14:18:01.333463 29538 executor.go:111] Tasks: 87 done / 88 total; 1 can run I0908 14:18:01.333809 29538 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0908 14:18:01.393254 29538 target.go:112] Terraform output is in /tmp/test1552617527/out --- PASS: TestMinimalWarmPool (0.50s) === RUN TestMinimalEtcd I0908 14:18:01.540994 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:01.541253 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:01.541315 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:01.541424 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:01.541478 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:01.541526 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:01.541556 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:01.541595 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:01.541628 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:01.568144 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:01.571214 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.571312 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.611241 29538 zones.go:93] ListHostedZonesPages { } W0908 14:18:01.697093 29538 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0908 14:18:01.697155 29538 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0908 14:18:01.699017 29538 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0908 14:18:01.699175 29538 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0908 14:18:01.699225 29538 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:01.711827 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:18:01.712227 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:01.712269 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:01.712364 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:01.712510 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:01.712723 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:01.713023 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:01.713160 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:01.713177 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:01.713279 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-etcd.example.com" }] }] } I0908 14:18:01.715334 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:01.723966 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:01.724226 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.724593 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.725662 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.726761 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.726897 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:01.727143 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:01.730485 29538 target.go:168] Cloudformation output is in /tmp/test937665846/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0908 14:18:01.741482 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:01.741704 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:01.741761 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:01.741806 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:01.741858 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:01.741937 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:01.741967 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:01.742006 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:01.742035 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:01.767627 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:01.770458 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.770549 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.812554 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:01.913419 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:01.913836 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:01.914136 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:01.914411 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:01.914517 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:01.914454 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:01.914731 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:01.918957 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:01.926569 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:01.926837 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.927422 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.928401 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.930032 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:01.930161 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:01.930675 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:01.984715 29538 target.go:112] Terraform output is in /tmp/test2090927559/out I0908 14:18:01.989621 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:01.989747 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:01.989785 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:01.989816 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:01.989851 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:01.989877 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:01.989893 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:01.989917 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:01.989934 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:02.004022 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:02.005658 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.005715 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.045574 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:02.142606 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:02.143027 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:02.143570 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:02.143606 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:02.143625 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:02.143732 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:02.143779 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0908 14:18:02.143847 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:02.143803 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:02.143903 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:02.145580 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:02.154231 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:02.154595 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.154830 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.155762 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.157136 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.157262 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:02.157530 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:02.160446 29538 target.go:168] Cloudformation output is in /tmp/test593030446/out --- PASS: TestMinimalGp3 (0.43s) === RUN TestMinimalLongClusterName I0908 14:18:02.171495 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:02.171733 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:02.171784 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:02.171829 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:02.171879 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:02.171960 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:02.171991 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:02.172030 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:02.172059 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:02.195827 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:02.198642 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.198732 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:18:02.238349 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:02.340873 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:02.341391 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:02.341596 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:02.341630 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:02.341713 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:02.341828 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:02.341881 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:02.343653 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:02.353444 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:02.353916 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.354164 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.355149 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.356527 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.356647 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:02.357030 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:02.414801 29538 target.go:112] Terraform output is in /tmp/test278681501/out I0908 14:18:02.428965 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:02.429110 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:02.429153 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:02.429194 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:02.429233 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:02.429263 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:02.429280 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:02.429313 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:02.429333 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:02.442776 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:02.444463 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.444527 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:18:02.485173 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:02.582964 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:18:02.583550 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:02.583603 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:02.583764 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:02.583798 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:02.584047 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:02.584132 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:02.584242 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:02.584270 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:02.584421 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } I0908 14:18:02.585932 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:02.591206 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:02.591558 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.593020 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.594417 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.598271 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.598424 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:02.598740 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:02.602006 29538 target.go:168] Cloudformation output is in /tmp/test1506764918/out --- PASS: TestMinimalLongClusterName (0.45s) === RUN TestExistingIAMCloudformation I0908 14:18:02.618740 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:02.618973 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:02.619027 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:02.619129 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:02.619185 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:02.619229 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:02.619258 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:02.619296 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:02.619328 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:02.636483 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:02.639382 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.639484 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.679924 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:02.781029 29538 executor.go:111] Tasks: 0 done / 75 total; 41 can run I0908 14:18:02.781688 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:02.781832 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:02.781859 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:02.781851 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:02.781904 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:02.781954 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:02.782036 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:02.781719 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:02.782106 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:02.784082 29538 executor.go:111] Tasks: 41 done / 75 total; 11 can run I0908 14:18:02.789167 29538 executor.go:111] Tasks: 52 done / 75 total; 21 can run I0908 14:18:02.791176 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.791516 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.794943 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.796000 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.796187 29538 executor.go:111] Tasks: 73 done / 75 total; 2 can run I0908 14:18:02.796459 29538 executor.go:111] Tasks: 75 done / 75 total; 0 can run I0908 14:18:02.798992 29538 target.go:168] Cloudformation output is in /tmp/test1298061416/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0908 14:18:02.807942 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:02.808179 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:02.808232 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:02.808280 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:02.808330 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:02.808410 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:02.808439 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:02.808479 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:02.808507 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:02.844221 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:02.847408 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.847497 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.847562 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.847613 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:02.887728 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:02.993476 29538 executor.go:111] Tasks: 0 done / 138 total; 49 can run W0908 14:18:02.993772 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:02.993822 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:02.993852 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:02.993965 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:02.994241 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:02.994471 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:02.994502 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:02.994665 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:02.997061 29538 executor.go:111] Tasks: 49 done / 138 total; 26 can run I0908 14:18:03.008084 29538 executor.go:111] Tasks: 75 done / 138 total; 58 can run I0908 14:18:03.008702 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.008957 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.009171 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.009476 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.010126 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.011370 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.012673 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.013911 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.014315 29538 executor.go:111] Tasks: 133 done / 138 total; 5 can run I0908 14:18:03.015733 29538 executor.go:111] Tasks: 138 done / 138 total; 0 can run I0908 14:18:03.111837 29538 target.go:112] Terraform output is in /tmp/test3271598273/out --- PASS: TestExistingSG (0.33s) === RUN TestBastionAdditionalUserData I0908 14:18:03.140243 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:03.140515 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:03.140582 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:03.140660 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:03.140721 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:03.140816 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:03.140847 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:03.140885 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:03.140915 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:03.175001 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:03.178141 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.178242 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.178309 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.220918 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:03.323792 29538 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0908 14:18:03.324090 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:03.324591 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:03.324653 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:03.324673 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.324643 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:03.324779 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:03.324904 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.327220 29538 executor.go:111] Tasks: 48 done / 119 total; 29 can run I0908 14:18:03.336826 29538 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0908 14:18:03.337061 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.337378 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.337698 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.338270 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.338907 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.339694 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.339891 29538 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0908 14:18:03.340480 29538 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0908 14:18:03.340713 29538 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0908 14:18:03.418788 29538 target.go:112] Terraform output is in /tmp/test2564932287/out --- PASS: TestBastionAdditionalUserData (0.30s) === RUN TestPrivateWeave I0908 14:18:03.436404 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:03.436556 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:03.436596 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:03.436630 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:03.436665 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:03.436725 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:03.436744 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:03.436769 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:03.436786 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:03.462358 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:03.465549 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.465675 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.465771 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.505509 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:03.626867 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0908 14:18:03.627471 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:03.627523 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:03.627661 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:03.627956 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:03.627987 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.627924 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:03.628177 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.631895 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:03.640508 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:03.640774 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.641238 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.641478 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.641960 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.642912 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.643479 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.643598 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:03.644028 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:03.644185 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:03.725195 29538 target.go:112] Terraform output is in /tmp/test135981680/out --- PASS: TestPrivateWeave (0.29s) === RUN TestPrivateFlannel I0908 14:18:03.730596 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:03.730768 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:03.730806 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:03.730843 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:03.730878 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:03.730936 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:03.730957 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:03.730980 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:03.730999 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:03.749238 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:03.750941 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.751002 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.751038 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.791514 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:03.917385 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0908 14:18:03.917808 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.917793 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:03.918278 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:03.918335 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:03.918386 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:03.918423 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:03.918592 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:03.920554 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:03.929158 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:03.929659 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.929919 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.930133 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.930222 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.931372 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.932000 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:03.932126 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:03.932577 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:03.932747 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:04.010188 29538 target.go:112] Terraform output is in /tmp/test3856034144/out --- PASS: TestPrivateFlannel (0.29s) === RUN TestPrivateCalico I0908 14:18:04.016423 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:04.016570 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:04.016609 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:04.016675 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:04.016710 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:04.016740 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:04.016757 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:04.016782 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:04.016800 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:04.048474 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:04.051823 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.051922 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.051984 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.093490 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:04.517539 29538 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0908 14:18:04.517807 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:04.518468 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:04.519944 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:04.518517 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:04.521653 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:04.521656 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:04.518691 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:04.525780 29538 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0908 14:18:04.531233 29538 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0908 14:18:04.531983 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.532250 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.532460 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.532541 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.533731 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.534562 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.534678 29538 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0908 14:18:04.535116 29538 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0908 14:18:04.535283 29538 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0908 14:18:04.617498 29538 target.go:112] Terraform output is in /tmp/test3582230439/out I0908 14:18:04.637781 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:04.638040 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:04.638114 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:04.638195 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:04.638284 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:04.638349 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:04.638404 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:04.638471 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:04.638516 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:04.671189 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:04.675439 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.675544 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.675606 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:04.716083 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:05.146037 29538 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0908 14:18:05.146637 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:05.146649 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:05.146817 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:05.146879 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:05.146916 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:05.147051 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:05.146982 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:05.148959 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:05.148998 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:05.149237 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:05.154252 29538 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0908 14:18:05.160043 29538 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0908 14:18:05.160282 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.160764 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.160984 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.161251 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.162089 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.163089 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.163221 29538 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0908 14:18:05.163575 29538 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0908 14:18:05.163718 29538 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0908 14:18:05.168239 29538 target.go:168] Cloudformation output is in /tmp/test4232507063/out --- PASS: TestPrivateCalico (1.18s) === RUN TestPrivateCilium I0908 14:18:05.192514 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:05.192763 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:05.192818 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:05.192865 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:05.192918 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:05.193000 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:05.193029 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:05.193065 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:05.193094 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:05.221812 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:05.224993 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.225084 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.225147 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.264233 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:05.414375 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0908 14:18:05.414774 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:05.414807 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:05.414805 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:05.414903 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:05.415298 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:05.415413 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:05.415502 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:05.421891 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:05.427763 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:05.428384 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.428583 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.428646 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.428961 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.429791 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.431274 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.431397 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:05.431806 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:05.431965 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:05.512503 29538 target.go:112] Terraform output is in /tmp/test1991206020/out I0908 14:18:05.523695 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:05.523830 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:05.523871 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:05.523904 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:05.523948 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:05.523980 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:05.523997 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:05.524022 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:05.524040 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:05.539934 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:05.541808 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.541867 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.541905 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.582466 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:05.730959 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0908 14:18:05.731434 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:05.731494 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:05.731638 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:05.731673 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:05.731901 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:05.731827 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" }] }] } I0908 14:18:05.731474 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0908 14:18:05.732172 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:05.732104 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:05.732227 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:05.734253 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:05.743196 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:05.743716 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.743899 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.744107 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.744186 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.745220 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.746110 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.746253 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:05.746614 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:05.746814 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:05.751486 29538 target.go:168] Cloudformation output is in /tmp/test1327316188/out --- PASS: TestPrivateCilium (0.58s) === RUN TestPrivateCilium2 I0908 14:18:05.774013 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:05.774254 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:05.774310 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:05.774360 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:05.774415 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:05.774497 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:05.774526 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:05.774567 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:05.774596 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:05.836714 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0908 14:18:05.840130 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.840227 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:05.840292 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:06.843040 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:06.989542 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0908 14:18:06.990110 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:06.990339 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:06.990379 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:06.990484 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:06.990166 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:06.990650 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:06.990649 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:06.993083 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:07.002332 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:07.002794 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.003070 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.003296 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.003382 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.003932 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.005580 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.005703 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:07.006152 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:07.006328 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:07.085058 29538 target.go:112] Terraform output is in /tmp/test136265448/out I0908 14:18:07.118319 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:07.118450 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:07.118488 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:07.118518 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:07.118551 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:07.118577 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:07.118593 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:07.118615 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:07.118633 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:07.139213 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0908 14:18:07.142578 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.142704 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.142770 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.184292 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:07.321392 29538 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0908 14:18:07.322135 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:07.322188 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:07.322234 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0908 14:18:07.322264 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:07.322268 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:07.322282 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:07.324379 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:07.322419 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:07.324440 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } W0908 14:18:07.324989 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:07.328507 29538 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0908 14:18:07.334498 29538 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0908 14:18:07.335091 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.335304 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.335425 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.335651 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.336565 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.337990 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.338170 29538 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0908 14:18:07.338581 29538 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0908 14:18:07.338747 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:07.342880 29538 target.go:168] Cloudformation output is in /tmp/test2612513083/out --- PASS: TestPrivateCilium2 (1.58s) === RUN TestPrivateCiliumAdvanced I0908 14:18:07.356100 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:07.356340 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:07.356394 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:07.356440 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:07.356490 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:07.356569 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:07.356599 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:07.356639 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:07.356667 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:07.395072 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:07.398437 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.398536 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.398601 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.440267 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:07.596662 29538 executor.go:111] Tasks: 0 done / 123 total; 53 can run W0908 14:18:07.597015 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:07.597254 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:07.597438 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:07.597496 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:07.601053 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:07.601123 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:07.601299 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:07.605130 29538 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0908 14:18:07.611852 29538 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0908 14:18:07.612347 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.612709 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.612871 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.612968 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.614112 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.615008 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.615138 29538 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0908 14:18:07.615644 29538 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0908 14:18:07.615798 29538 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0908 14:18:07.699019 29538 target.go:112] Terraform output is in /tmp/test3114101497/out I0908 14:18:07.704474 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:07.704606 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:07.704642 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:07.704673 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:07.704707 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:07.704733 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:07.704750 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:07.704772 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:07.704790 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:07.723329 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:07.725242 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.725298 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.725333 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.764735 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:07.917267 29538 executor.go:111] Tasks: 0 done / 123 total; 53 can run W0908 14:18:07.917526 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:07.917710 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:07.917851 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:07.918109 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:07.918136 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:07.920871 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:07.921054 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:07.921126 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:07.921157 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:07.921326 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0908 14:18:07.925762 29538 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0908 14:18:07.932532 29538 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0908 14:18:07.932956 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.933172 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.933389 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.933492 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.934572 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.935324 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.935472 29538 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0908 14:18:07.935861 29538 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0908 14:18:07.936011 29538 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0908 14:18:07.940220 29538 target.go:168] Cloudformation output is in /tmp/test2317480489/out --- PASS: TestPrivateCiliumAdvanced (0.60s) === RUN TestPrivateCanal I0908 14:18:07.959470 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:07.959695 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:07.959746 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:07.959791 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:07.959849 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:07.959934 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:07.959967 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:07.960009 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:07.960038 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:07.992052 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0908 14:18:07.995155 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.995247 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:07.995307 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:08.756626 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:09.173280 29538 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0908 14:18:09.173779 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:09.174092 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:09.174221 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:09.174257 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:09.175023 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:09.175452 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:09.176322 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:09.181469 29538 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0908 14:18:09.187027 29538 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0908 14:18:09.187239 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.187452 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.187660 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.187894 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.189337 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.190917 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:09.191069 29538 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0908 14:18:09.191527 29538 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0908 14:18:09.191707 29538 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0908 14:18:09.271249 29538 target.go:112] Terraform output is in /tmp/test4126930448/out --- PASS: TestPrivateCanal (1.35s) === RUN TestPrivateKopeio I0908 14:18:09.305046 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:09.305193 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:09.305231 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:09.305263 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:09.305297 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:09.305356 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:09.305376 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:09.305401 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:09.305420 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:09.322045 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:09.323830 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.323888 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.323924 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.363712 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:09.477815 29538 executor.go:111] Tasks: 0 done / 123 total; 47 can run W0908 14:18:09.478047 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:09.478080 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:09.478137 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:09.478391 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:09.478436 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:09.478532 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:09.480047 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:09.485264 29538 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0908 14:18:09.491200 29538 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0908 14:18:09.491474 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.491876 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.492145 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.492421 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.493354 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.493874 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.494021 29538 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0908 14:18:09.494513 29538 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0908 14:18:09.576565 29538 target.go:112] Terraform output is in /tmp/test719068776/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I0908 14:18:09.594482 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:09.594630 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:09.594672 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:09.594704 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:09.594736 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:09.594795 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:09.594813 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:09.594836 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:09.594853 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:09.612462 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:09.614338 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.614396 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.614433 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.656702 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:09.764421 29538 executor.go:111] Tasks: 0 done / 105 total; 45 can run I0908 14:18:09.764929 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:09.764972 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:09.765077 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:09.765348 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:09.765400 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:09.765558 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:09.765693 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:09.767298 29538 executor.go:111] Tasks: 45 done / 105 total; 25 can run I0908 14:18:09.773570 29538 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0908 14:18:09.773930 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.774137 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.774212 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.774330 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.775273 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.776742 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.776891 29538 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0908 14:18:09.777374 29538 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0908 14:18:09.852211 29538 target.go:112] Terraform output is in /tmp/test3511935092/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I0908 14:18:09.866932 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:09.867083 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:09.867123 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:09.867154 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:09.867188 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:09.867250 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:09.867267 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:09.867305 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:09.867322 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:09.883778 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:09.885546 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.885604 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.885641 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:09.926062 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:10.025122 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:18:10.025744 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:10.025798 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:10.025940 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:10.025963 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.025917 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:10.026094 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:10.026107 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.028353 29538 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0908 14:18:10.028554 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:10.037476 29538 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0908 14:18:10.037936 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.038122 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.038346 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.038518 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.039152 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.040336 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.040461 29538 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0908 14:18:10.040922 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:18:10.112310 29538 target.go:112] Terraform output is in /tmp/test2613285519/out --- PASS: TestPrivateSharedSubnet (0.25s) === RUN TestPrivateSharedIP I0908 14:18:10.117595 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:10.117746 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:10.117785 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:10.117819 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:10.117853 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:10.117914 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:10.117934 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:10.117960 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:10.117978 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:10.134209 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:10.139265 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.139655 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.139728 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.181092 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:10.283125 29538 executor.go:111] Tasks: 0 done / 113 total; 45 can run W0908 14:18:10.284071 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.284048 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:10.284172 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:10.284297 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.284327 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:18:10.284768 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.284786 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:10.288785 29538 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0908 14:18:10.289063 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:10.294777 29538 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0908 14:18:10.295319 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.295466 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.295681 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.295940 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.296946 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.298206 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.298364 29538 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0908 14:18:10.298878 29538 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0908 14:18:10.299052 29538 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0908 14:18:10.376661 29538 target.go:112] Terraform output is in /tmp/test3919587557/out I0908 14:18:10.381885 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:10.382020 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:10.382058 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:10.382091 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:10.382123 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:10.382151 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:10.382167 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:10.382191 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:10.382208 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:10.411493 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:10.414941 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.415087 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.415152 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.466095 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:10.571266 29538 executor.go:111] Tasks: 0 done / 113 total; 45 can run W0908 14:18:10.571597 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:10.571778 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:10.571882 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:10.571921 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:10.571957 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:10.572049 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:10.571893 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:10.572147 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:10.572167 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:10.572262 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:10.578192 29538 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0908 14:18:10.578927 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:10.583659 29538 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0908 14:18:10.584288 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.584436 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.584504 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.584749 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.586511 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.587692 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:10.587872 29538 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0908 14:18:10.588442 29538 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0908 14:18:10.588692 29538 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0908 14:18:10.592868 29538 target.go:168] Cloudformation output is in /tmp/test3883012112/out --- PASS: TestPrivateSharedIP (0.49s) === RUN TestPrivateDns1 I0908 14:18:10.605537 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:10.605767 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:10.605821 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:10.605872 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:10.605925 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:10.606006 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:10.606036 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:10.606074 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:10.606103 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:10.641983 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.642083 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.642145 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.802490 29538 executor.go:111] Tasks: 0 done / 117 total; 46 can run I0908 14:18:10.802983 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:10.803168 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:10.803280 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:10.803310 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:10.806096 29538 executor.go:111] Tasks: 46 done / 117 total; 24 can run I0908 14:18:10.806814 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0908 14:18:10.806858 29538 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0908 14:18:10.806983 29538 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0908 14:18:10.807003 29538 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0908 14:18:10.807116 29538 executor.go:111] Tasks: 70 done / 117 total; 35 can run I0908 14:18:10.807817 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.808020 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.814641 29538 executor.go:111] Tasks: 105 done / 117 total; 5 can run W0908 14:18:10.814875 29538 api.go:53] PageSize not implemented W0908 14:18:10.814985 29538 api.go:53] PageSize not implemented I0908 14:18:10.820201 29538 executor.go:111] Tasks: 110 done / 117 total; 5 can run I0908 14:18:10.820396 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.820620 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.821532 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.823284 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.823414 29538 executor.go:111] Tasks: 115 done / 117 total; 2 can run I0908 14:18:10.823766 29538 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0908 14:18:10.915814 29538 target.go:112] Terraform output is in /tmp/test1298481804/out --- PASS: TestPrivateDns1 (0.35s) === RUN TestPrivateDns2 I0908 14:18:10.952258 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:10.952408 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:10.952443 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:10.952474 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:10.952508 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:10.952566 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:10.952584 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:10.952608 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:10.952624 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:10.970745 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.970804 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:10.970840 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.104241 29538 executor.go:111] Tasks: 0 done / 113 total; 44 can run W0908 14:18:11.104783 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:11.105206 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:11.105260 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:11.105373 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:11.107570 29538 executor.go:111] Tasks: 44 done / 113 total; 22 can run I0908 14:18:11.107913 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:11.108039 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0908 14:18:11.108081 29538 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0908 14:18:11.108174 29538 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0908 14:18:11.108196 29538 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0908 14:18:11.108333 29538 executor.go:111] Tasks: 66 done / 113 total; 35 can run I0908 14:18:11.108795 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.108967 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.112251 29538 executor.go:111] Tasks: 101 done / 113 total; 5 can run W0908 14:18:11.112423 29538 api.go:53] PageSize not implemented W0908 14:18:11.112472 29538 api.go:53] PageSize not implemented I0908 14:18:11.120860 29538 executor.go:111] Tasks: 106 done / 113 total; 5 can run I0908 14:18:11.121109 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.121329 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.122410 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.123768 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.123973 29538 executor.go:111] Tasks: 111 done / 113 total; 2 can run I0908 14:18:11.124319 29538 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0908 14:18:11.198413 29538 target.go:112] Terraform output is in /tmp/test545737464/out --- PASS: TestPrivateDns2 (0.25s) === RUN TestDiscoveryFeatureGate I0908 14:18:11.203715 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:11.203859 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:11.203898 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:11.203930 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:11.203985 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:11.204042 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:11.204060 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:11.204083 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:11.204101 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:11.218487 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0908 14:18:11.220283 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.220339 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.262424 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:11.370983 29538 executor.go:111] Tasks: 0 done / 90 total; 46 can run W0908 14:18:11.371429 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:11.371469 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:11.373689 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:11.374216 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:11.374262 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:11.374383 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:11.379118 29538 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0908 14:18:11.384347 29538 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0908 14:18:11.384632 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.384911 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.386555 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.387906 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.388026 29538 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0908 14:18:11.388341 29538 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0908 14:18:11.444577 29538 target.go:112] Terraform output is in /tmp/test3472688826/out --- PASS: TestDiscoveryFeatureGate (0.25s) === RUN TestVFSServiceAccountIssuerDiscovery I0908 14:18:11.450122 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:11.450270 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:11.450316 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:11.450350 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:11.450383 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:11.450442 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:11.450459 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:11.450482 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:11.450498 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:11.464579 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:11.466271 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.466354 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.506362 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:11.605481 29538 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0908 14:18:11.605810 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:11.606049 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:11.606155 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:11.606189 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:11.606314 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:11.606462 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:11.613825 29538 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0908 14:18:11.621762 29538 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0908 14:18:11.622123 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.622315 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.623431 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.624535 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.624660 29538 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0908 14:18:11.624966 29538 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0908 14:18:11.681438 29538 target.go:112] Terraform output is in /tmp/test1994355573/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.35s) === RUN TestAWSLBController I0908 14:18:11.804130 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:11.804377 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:11.804434 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:11.804483 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:11.804541 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:11.804632 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:11.804662 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:11.804699 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:11.804731 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:11.840345 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:11.843342 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.843441 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:11.882845 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:12.497098 29538 executor.go:111] Tasks: 0 done / 94 total; 49 can run W0908 14:18:12.497404 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:12.497755 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:12.497948 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:12.497967 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:12.498037 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:12.500250 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:12.506247 29538 executor.go:111] Tasks: 49 done / 94 total; 22 can run I0908 14:18:12.512340 29538 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0908 14:18:12.512847 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.513069 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.515761 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.519124 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.519266 29538 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0908 14:18:12.519673 29538 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0908 14:18:12.575943 29538 target.go:112] Terraform output is in /tmp/test1860816567/out --- PASS: TestAWSLBController (0.94s) === RUN TestManyAddons I0908 14:18:12.741570 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:12.741743 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:12.741789 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:12.741833 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:12.741880 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:12.741946 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:12.741978 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:12.742017 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:12.742041 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:12.757049 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:12.759082 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.759151 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:12.799659 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:13.590820 29538 executor.go:111] Tasks: 0 done / 93 total; 50 can run W0908 14:18:13.591487 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:13.591514 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:13.592912 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:13.592955 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:13.593086 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:13.594290 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:13.598604 29538 executor.go:111] Tasks: 50 done / 93 total; 19 can run I0908 14:18:13.603935 29538 executor.go:111] Tasks: 69 done / 93 total; 22 can run I0908 14:18:13.604515 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.604751 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.605930 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.607074 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.607195 29538 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0908 14:18:13.607533 29538 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0908 14:18:13.664456 29538 target.go:112] Terraform output is in /tmp/test2734361349/out --- PASS: TestManyAddons (0.98s) === RUN TestManyAddonsCCMIRSA I0908 14:18:13.724439 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:13.724614 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:13.724655 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:13.724687 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:13.724720 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:13.724774 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:13.724792 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:13.724815 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:13.724835 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:13.740643 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:13.742722 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.742784 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:13.781887 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:14.644154 29538 executor.go:111] Tasks: 0 done / 109 total; 59 can run I0908 14:18:14.644928 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:14.644993 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:14.644995 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:18:14.645141 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:14.645156 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:14.645196 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:14.649218 29538 executor.go:111] Tasks: 59 done / 109 total; 26 can run I0908 14:18:14.659989 29538 executor.go:111] Tasks: 85 done / 109 total; 22 can run I0908 14:18:14.660350 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.660640 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.661980 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.663558 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.663703 29538 executor.go:111] Tasks: 107 done / 109 total; 2 can run I0908 14:18:14.664056 29538 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0908 14:18:14.735687 29538 target.go:112] Terraform output is in /tmp/test2943992711/out --- PASS: TestManyAddonsCCMIRSA (1.05s) === RUN TestManyAddonsCCMIRSA23 I0908 14:18:14.777866 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:14.778018 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:14.778056 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:14.778087 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:14.778120 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:14.778175 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:14.778193 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:14.778216 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:14.778232 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:14.793699 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0908 14:18:14.795722 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.795780 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:14.836718 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:15.710593 29538 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0908 14:18:15.711215 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:15.711390 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:15.711692 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:15.711720 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:15.711819 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:15.711905 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:15.719573 29538 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0908 14:18:15.725310 29538 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0908 14:18:15.725903 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.726025 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.727206 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.728357 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.728484 29538 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0908 14:18:15.728791 29538 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0908 14:18:15.803338 29538 target.go:112] Terraform output is in /tmp/test217292935/out --- PASS: TestManyAddonsCCMIRSA23 (1.05s) === RUN TestManyAddonsCCMIRSA24 I0908 14:18:15.826005 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:15.826166 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:15.826206 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:15.826241 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:15.826275 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:15.826335 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:15.826356 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:15.826385 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:15.826405 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:15.845129 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:15.847552 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.847660 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:15.887171 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:16.766753 29538 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0908 14:18:16.767032 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:16.767173 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:16.767202 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:16.767287 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:16.768202 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:16.768360 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:16.776966 29538 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0908 14:18:16.787322 29538 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0908 14:18:16.787551 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.787855 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.788767 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.790406 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.790549 29538 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0908 14:18:16.790916 29538 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0908 14:18:16.860086 29538 target.go:112] Terraform output is in /tmp/test1927213157/out --- PASS: TestManyAddonsCCMIRSA24 (1.05s) === RUN TestCCM I0908 14:18:16.880000 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:16.880155 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:16.880194 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:16.880224 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:16.880257 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:16.880315 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:16.880334 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:16.880359 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:16.880376 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:16.903233 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:16.906757 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.906855 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:16.946299 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:17.755161 29538 executor.go:111] Tasks: 0 done / 94 total; 51 can run W0908 14:18:17.756586 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:17.756773 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:17.757228 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:17.757259 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:17.757390 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:17.757566 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:17.764270 29538 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0908 14:18:17.775114 29538 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0908 14:18:17.775774 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.776007 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.777149 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.778634 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.778752 29538 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0908 14:18:17.779099 29538 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0908 14:18:17.836994 29538 target.go:112] Terraform output is in /tmp/test913782239/out --- PASS: TestCCM (0.97s) === RUN TestExternalDNS I0908 14:18:17.853321 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:17.853465 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:17.853511 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:17.853544 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:17.853581 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:17.853637 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:17.853655 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:17.853677 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:17.853695 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:17.867674 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:17.869275 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.869333 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:17.908320 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:18.008787 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:18.009479 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:18.009561 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:18.009885 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:18.010623 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:18.010941 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:18.009773 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:18.017172 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:18.023072 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:18.023426 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.023697 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.024841 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.025815 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.025938 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:18.026231 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:18.080408 29538 target.go:112] Terraform output is in /tmp/test2222656952/out I0908 14:18:18.090977 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:18.091118 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:18.091156 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:18.091186 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:18.091218 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:18.091244 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:18.091264 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:18.091288 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:18.091305 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:18.105033 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:18.106684 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.106742 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.146802 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:18.249913 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:18.250548 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:18.250714 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:18.250774 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:18.250769 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:18.250914 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:18.250937 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:18.251826 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0908 14:18:18.250812 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:18.253054 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:18.258131 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:18.264104 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:18.264551 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.264760 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.265740 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.267719 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.267885 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:18.268196 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:18.271356 29538 target.go:168] Cloudformation output is in /tmp/test3924886854/out --- PASS: TestExternalDNS (0.43s) === RUN TestExternalDNSIRSA I0908 14:18:18.281864 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:18.282099 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:18.282152 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:18.282199 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:18.282247 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:18.282330 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:18.282360 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:18.282402 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:18.282431 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:18.312532 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:18.315556 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.315660 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.355232 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:18.472080 29538 executor.go:111] Tasks: 0 done / 90 total; 46 can run W0908 14:18:18.472443 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:18.472470 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:18.473071 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:18.473251 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:18.473288 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:18.473410 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:18.476642 29538 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0908 14:18:18.484765 29538 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0908 14:18:18.485176 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.485397 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.486465 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.488023 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.488230 29538 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0908 14:18:18.488670 29538 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0908 14:18:18.549328 29538 target.go:112] Terraform output is in /tmp/test3384311910/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestKarpenter I0908 14:18:18.554595 29538 featureflag.go:162] FeatureFlag "Karpenter"=true I0908 14:18:18.554614 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:18.554741 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:18.554776 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:18.554806 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:18.554838 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:18.554892 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:18.554911 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:18.554958 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:18.554974 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:18.577846 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:18.580913 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.581006 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.581067 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.621218 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:18.715458 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.715626 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.715742 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.809287 29538 executor.go:111] Tasks: 0 done / 101 total; 50 can run W0908 14:18:18.810879 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:18.811006 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:18.811950 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:18.812317 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:18.812366 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:18.812514 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:18.813764 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:18.815540 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:18.817451 29538 executor.go:111] Tasks: 50 done / 101 total; 24 can run I0908 14:18:18.822940 29538 executor.go:111] Tasks: 74 done / 101 total; 25 can run I0908 14:18:18.823260 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.823426 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.823607 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.823844 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.824488 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.824784 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.825334 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.825809 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.825940 29538 executor.go:111] Tasks: 99 done / 101 total; 2 can run I0908 14:18:18.827184 29538 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0908 14:18:18.900044 29538 target.go:112] Terraform output is in /tmp/test3113055986/out I0908 14:18:18.938406 29538 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.38s) === RUN TestSharedSubnet I0908 14:18:18.938483 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:18.938622 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:18.938658 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:18.938689 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:18.938719 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:18.938775 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:18.938792 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:18.938814 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:18.938833 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:18.955104 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:18.956719 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.956777 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:18.997513 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:19.097037 29538 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0908 14:18:19.097708 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:19.097780 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:19.097948 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:19.098132 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:19.098168 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:19.098296 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:19.099729 29538 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0908 14:18:19.100338 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:19.104899 29538 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0908 14:18:19.105231 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.105462 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.106324 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.107776 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.107902 29538 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0908 14:18:19.108207 29538 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0908 14:18:19.160133 29538 target.go:112] Terraform output is in /tmp/test603000723/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I0908 14:18:19.181061 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:19.181220 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:19.181255 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:19.181292 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:19.181331 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:19.181400 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:19.181420 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:19.181450 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:19.181472 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:19.195415 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:19.197079 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.197146 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.237986 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:19.333291 29538 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0908 14:18:19.333674 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:19.333741 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:19.333716 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:19.333811 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:19.334281 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:19.334246 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:19.336131 29538 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0908 14:18:19.336539 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:19.343911 29538 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0908 14:18:19.345099 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.345419 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.346614 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.347441 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.347565 29538 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0908 14:18:19.347893 29538 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0908 14:18:19.397582 29538 target.go:112] Terraform output is in /tmp/test3782824988/out --- PASS: TestSharedVPC (0.23s) === RUN TestExistingIAM I0908 14:18:19.408628 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:19.408769 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:19.408808 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:19.408840 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:19.408874 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:19.408930 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:19.408947 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:19.408971 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:19.408987 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:19.446666 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:19.449877 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.449982 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.450046 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.450116 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.489880 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:19.591925 29538 executor.go:111] Tasks: 0 done / 93 total; 47 can run W0908 14:18:19.592420 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:19.592545 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:19.592640 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:19.592697 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:19.592846 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:19.593018 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:19.593050 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:19.593130 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:19.594716 29538 executor.go:111] Tasks: 47 done / 93 total; 15 can run I0908 14:18:19.599872 29538 executor.go:111] Tasks: 62 done / 93 total; 27 can run I0908 14:18:19.600388 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.600592 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.600897 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.601177 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.603565 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.604564 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.605982 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.606776 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:19.606885 29538 executor.go:111] Tasks: 89 done / 93 total; 4 can run I0908 14:18:19.608179 29538 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0908 14:18:19.680564 29538 target.go:112] Terraform output is in /tmp/test4143938665/out --- PASS: TestExistingIAM (0.29s) === RUN TestPhaseNetwork I0908 14:18:19.697875 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:19.698029 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:19.698069 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:19.698101 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:19.698135 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:19.698163 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:19.698180 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:19.698207 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:19.698225 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:19.714409 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:19.716167 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.716226 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.716263 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.756787 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:19.861125 29538 executor.go:111] Tasks: 0 done / 116 total; 46 can run I0908 14:18:19.861489 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:19.861626 29538 executor.go:111] Tasks: 46 done / 116 total; 28 can run I0908 14:18:19.862170 29538 executor.go:111] Tasks: 74 done / 116 total; 36 can run I0908 14:18:19.862589 29538 executor.go:111] Tasks: 110 done / 116 total; 5 can run I0908 14:18:19.862781 29538 executor.go:111] Tasks: 115 done / 116 total; 1 can run I0908 14:18:19.862930 29538 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0908 14:18:19.872062 29538 target.go:112] Terraform output is in /tmp/test1531439483/out --- PASS: TestPhaseNetwork (0.18s) === RUN TestExternalLoadBalancer I0908 14:18:19.878724 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:19.878961 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:19.879017 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:19.879068 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:19.879135 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:19.879225 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:19.879255 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:19.879291 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:19.879320 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:19.904694 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:19.907570 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.907670 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:19.948881 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:20.048399 29538 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0908 14:18:20.048830 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:20.048900 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:20.048936 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:20.049025 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:20.049369 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:20.049355 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:20.052240 29538 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0908 14:18:20.062561 29538 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0908 14:18:20.062969 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.063141 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.064181 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.065422 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.065544 29538 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0908 14:18:20.065883 29538 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0908 14:18:20.120950 29538 target.go:112] Terraform output is in /tmp/test1929752643/out I0908 14:18:20.125902 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:20.126041 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:20.126077 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:20.126108 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:20.126142 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:20.126169 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:20.126186 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:20.126210 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:20.126228 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:20.144302 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:20.147211 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.147338 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.186951 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:20.284511 29538 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0908 14:18:20.284872 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:20.285285 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:20.285268 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:20.285380 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:20.285438 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:20.285400 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:20.285507 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:20.285481 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:20.285614 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:20.287264 29538 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0908 14:18:20.292428 29538 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0908 14:18:20.292875 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.293086 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.294193 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.295466 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.295636 29538 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0908 14:18:20.295896 29538 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0908 14:18:20.298863 29538 target.go:168] Cloudformation output is in /tmp/test591655603/out --- PASS: TestExternalLoadBalancer (0.43s) === RUN TestPhaseIAM integration_test.go:750: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:758: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0908 14:18:20.313271 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:20.313511 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:20.313561 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:20.313614 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:20.313664 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:20.313744 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:20.313775 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:20.313825 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:20.313853 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:20.347049 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:20.349981 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350069 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350128 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350182 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350230 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350274 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.350331 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.389620 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:20.491785 29538 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0908 14:18:20.492214 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:20.492449 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:20.492721 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:20.493604 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:20.494568 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:20.494605 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:20.494709 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:20.495181 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:20.500234 29538 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0908 14:18:20.505998 29538 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0908 14:18:20.506379 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.506546 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.506718 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.506931 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.507859 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.510806 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.511919 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.512513 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.512792 29538 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0908 14:18:20.514233 29538 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0908 14:18:20.592262 29538 target.go:112] Terraform output is in /tmp/test3482778397/out I0908 14:18:20.607256 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:20.607409 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:20.607452 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:20.607485 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:20.607519 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:20.607551 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:20.607568 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:20.607593 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:20.607610 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:20.638463 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:20.641722 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.641816 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.641874 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.641934 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.641989 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.642038 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.642087 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.682527 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:20.786609 29538 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0908 14:18:20.787171 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:20.787360 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:20.787446 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:20.787469 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:20.787640 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:20.787660 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:20.787741 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:20.787688 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:20.787781 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:20.787891 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:20.787837 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" }] }] } I0908 14:18:20.789910 29538 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0908 14:18:20.798407 29538 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0908 14:18:20.799479 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.799737 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.799998 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.800265 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.801964 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.802103 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.802422 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.803029 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.803173 29538 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0908 14:18:20.803506 29538 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0908 14:18:20.808939 29538 target.go:168] Cloudformation output is in /tmp/test1112506768/out --- PASS: TestMixedInstancesASG (0.51s) === RUN TestMixedInstancesSpotASG I0908 14:18:20.827710 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:20.827942 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:20.827994 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:20.828044 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:20.828094 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:20.828174 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:20.828203 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:20.828240 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:20.828270 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:20.865605 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:20.868791 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.868887 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.868947 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.869005 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.869059 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.869106 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.869152 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:20.910173 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:21.013689 29538 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0908 14:18:21.014139 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:21.014497 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.014635 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:21.014764 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:21.014679 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:21.014819 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:21.014867 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:21.014894 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:21.017189 29538 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0908 14:18:21.022589 29538 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0908 14:18:21.022943 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.023297 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.023459 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.024279 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.026633 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.030261 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.030394 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.030493 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.030573 29538 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0908 14:18:21.031073 29538 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0908 14:18:21.110739 29538 target.go:112] Terraform output is in /tmp/test2369325848/out I0908 14:18:21.116240 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:21.116369 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:21.116407 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:21.116438 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:21.116474 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:21.116502 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:21.116519 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:21.116545 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:21.116563 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:21.144935 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:21.148822 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.148927 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.148991 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.149049 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.149101 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.149152 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.149201 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.189067 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:21.291870 29538 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0908 14:18:21.292151 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.292315 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.292351 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.292552 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:21.292909 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:21.292884 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.292944 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:18:21.293057 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:21.293058 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:21.293081 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.293205 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0908 14:18:21.295301 29538 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0908 14:18:21.303963 29538 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0908 14:18:21.304286 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.304540 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.304741 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.304973 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.309082 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.310756 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.311465 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.311615 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.311731 29538 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0908 14:18:21.312088 29538 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0908 14:18:21.316560 29538 target.go:168] Cloudformation output is in /tmp/test1859959452/out --- PASS: TestMixedInstancesSpotASG (0.51s) === RUN TestContainerd I0908 14:18:21.334148 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:21.334389 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:21.334441 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:21.334537 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:21.334593 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:21.334635 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:21.334665 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:21.334703 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:21.334731 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:21.364228 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:21.367024 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.367116 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.406790 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:21.505673 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:21.506186 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:21.506495 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:21.506578 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.506678 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:21.506747 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:21.506716 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.506782 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:21.506893 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:21.506874 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0908 14:18:21.510314 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:21.519937 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:21.520446 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.520707 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.521907 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.522890 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.523101 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:21.523485 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:21.526556 29538 target.go:168] Cloudformation output is in /tmp/test3425035886/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0908 14:18:21.540297 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:21.540524 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:21.540580 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:21.540710 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:21.540766 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:21.540809 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:21.540842 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:21.540882 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:21.540912 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:21.573826 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:21.576995 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.577088 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.616631 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:21.720392 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:18:21.720824 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:21.720881 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:21.721027 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:21.721270 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:21.721445 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:21.721487 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.721632 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:21.721663 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.721774 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0908 14:18:21.723847 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:21.729416 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:21.729830 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.730033 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.731155 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.732573 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.732712 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:21.732992 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:21.736024 29538 target.go:168] Cloudformation output is in /tmp/test2309215754/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I0908 14:18:21.747143 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:21.747394 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:21.747448 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:21.747537 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:21.747588 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:21.747629 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:21.747658 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:21.747697 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:21.747724 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:21.779699 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:21.782822 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.782918 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.822759 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:21.927103 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0908 14:18:21.927593 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:21.927909 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:21.928240 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:21.928214 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.928302 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:21.928432 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0908 14:18:21.928483 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:21.928514 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:21.928633 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" },{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" }] }] } I0908 14:18:21.930358 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:21.935718 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:21.936120 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.936308 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.937492 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.940733 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:21.941074 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:21.942211 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:21.946289 29538 target.go:168] Cloudformation output is in /tmp/test1520050751/out --- PASS: TestDockerCustom (0.21s) === RUN TestAPIServerNodes I0908 14:18:21.956794 29538 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0908 14:18:21.956842 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:21.957054 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:21.957106 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:21.957200 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:21.957256 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:21.957327 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:21.957357 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:21.957398 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:21.957427 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:21.985586 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0908 14:18:21.988594 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.988691 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:21.988760 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.030774 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:22.190267 29538 executor.go:111] Tasks: 0 done / 97 total; 47 can run W0908 14:18:22.190523 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:22.190827 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:22.190863 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:22.190969 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0908 14:18:22.190993 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:22.191096 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } W0908 14:18:22.190968 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:22.191278 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:22.191315 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:22.191411 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:22.193103 29538 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0908 14:18:22.198129 29538 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0908 14:18:22.198473 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.198730 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.198957 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.200859 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.201371 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.205696 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.205871 29538 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0908 14:18:22.206227 29538 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0908 14:18:22.209880 29538 target.go:168] Cloudformation output is in /tmp/test3560000712/out I0908 14:18:22.242785 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:22.243007 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:22.243063 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:22.243114 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:22.243166 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:22.243206 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:22.243233 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:22.243269 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:22.243300 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:22.270275 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0908 14:18:22.273268 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.273360 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.273426 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.314818 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:22.472950 29538 executor.go:111] Tasks: 0 done / 97 total; 47 can run W0908 14:18:22.473406 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:22.473463 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:22.473477 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:22.473572 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:22.473836 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:22.473875 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:22.474039 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:22.476730 29538 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0908 14:18:22.484018 29538 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0908 14:18:22.484647 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.484843 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.485058 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.487676 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.488058 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.489371 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.489501 29538 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0908 14:18:22.489911 29538 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0908 14:18:22.556944 29538 target.go:112] Terraform output is in /tmp/test3809713595/out I0908 14:18:22.564573 29538 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.61s) === RUN TestNTHQueueProcessor I0908 14:18:22.564652 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:22.564789 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:22.564826 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:22.564854 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:22.564886 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:22.564942 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:22.564960 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:22.564982 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:22.565000 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:22.589847 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:22.592862 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:22.592968 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:23.824209 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:23.948438 29538 executor.go:111] Tasks: 0 done / 99 total; 45 can run W0908 14:18:23.949154 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:23.949997 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:23.950595 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:23.950660 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:23.950796 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:23.951508 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:23.956920 29538 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0908 14:18:23.962462 29538 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0908 14:18:23.963172 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:23.963500 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:23.964356 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:23.965818 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:23.966022 29538 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0908 14:18:23.966509 29538 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0908 14:18:23.966710 29538 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0908 14:18:24.035050 29538 target.go:112] Terraform output is in /tmp/test3723901591/out I0908 14:18:24.040562 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:24.040709 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:24.040745 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:24.040775 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:24.040807 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:24.040832 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:24.040849 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:24.040873 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:24.040892 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:24.054569 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:24.056234 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.056288 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.097732 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0908 14:18:24.221940 29538 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0908 14:18:24.222330 29538 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0908 14:18:24.222369 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:24.222471 29538 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0908 14:18:24.222560 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0908 14:18:24.222737 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0908 14:18:24.222747 29538 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0908 14:18:24.222874 29538 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0908 14:18:24.222806 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:24.222979 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:24.226584 29538 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0908 14:18:24.236021 29538 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0908 14:18:24.236672 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.236928 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.238028 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.239191 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.239336 29538 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0908 14:18:24.239611 29538 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0908 14:18:24.239892 29538 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0908 14:18:24.244229 29538 target.go:168] Cloudformation output is in /tmp/test2211305829/out --- PASS: TestNTHQueueProcessor (1.69s) === RUN TestCustomIRSA I0908 14:18:24.255439 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:24.255678 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:24.255730 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:24.255775 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:24.255825 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:24.255908 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:24.255939 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:24.255977 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:24.256005 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:24.289535 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:24.292919 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.293019 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.332625 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:24.885387 29538 executor.go:111] Tasks: 0 done / 98 total; 50 can run I0908 14:18:24.886616 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0908 14:18:24.886647 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:24.886687 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:24.886672 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:24.886799 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:24.886799 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:24.893807 29538 executor.go:111] Tasks: 50 done / 98 total; 25 can run I0908 14:18:24.899094 29538 executor.go:111] Tasks: 75 done / 98 total; 21 can run I0908 14:18:24.899298 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.899665 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.900748 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.901575 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:24.901696 29538 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0908 14:18:24.902045 29538 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0908 14:18:24.960724 29538 target.go:112] Terraform output is in /tmp/test2568303036/out --- PASS: TestCustomIRSA (0.74s) === RUN TestCustomIRSA119 I0908 14:18:24.993147 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:24.993301 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:24.993342 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:24.993375 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:24.993407 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:24.993463 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:24.993480 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:24.993503 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:24.993521 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:25.009589 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0908 14:18:25.012315 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.012414 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.053912 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.152458 29538 executor.go:111] Tasks: 0 done / 96 total; 48 can run I0908 14:18:25.153414 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:25.153456 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:25.153575 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0908 14:18:25.153643 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:25.153704 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:25.153801 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.155389 29538 executor.go:111] Tasks: 48 done / 96 total; 25 can run I0908 14:18:25.165678 29538 executor.go:111] Tasks: 73 done / 96 total; 21 can run I0908 14:18:25.165928 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.166327 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.167519 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.168248 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.168366 29538 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0908 14:18:25.168687 29538 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0908 14:18:25.230680 29538 target.go:112] Terraform output is in /tmp/test2924349477/out --- PASS: TestCustomIRSA119 (0.24s) === RUN TestClusterNameDigit I0908 14:18:25.236190 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:25.236336 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:25.236374 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:25.236405 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:25.236438 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:25.236492 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:25.236508 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:25.236533 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:25.236550 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:25.251885 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:25.253693 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.253750 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.295874 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.396039 29538 executor.go:111] Tasks: 0 done / 93 total; 47 can run W0908 14:18:25.396325 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target W0908 14:18:25.396533 29538 warmpool.go:124] ASG warm pool is not supported by the terraform target I0908 14:18:25.396800 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.396874 29538 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0908 14:18:25.396915 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:25.397062 29538 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0908 14:18:25.399603 29538 executor.go:111] Tasks: 47 done / 93 total; 23 can run I0908 14:18:25.409430 29538 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0908 14:18:25.410058 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.410368 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.411138 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.412623 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.412747 29538 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0908 14:18:25.413047 29538 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0908 14:18:25.473355 29538 target.go:112] Terraform output is in /tmp/test2215362861/out --- PASS: TestClusterNameDigit (0.24s) === RUN TestLifecycleMinimalAWS I0908 14:18:25.479197 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:25.479343 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:25.479378 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:25.479408 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:25.479443 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:25.479471 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:25.479490 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:25.479516 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:25.479534 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:25.479578 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0908 14:18:25.493121 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:25.494734 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.494792 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:18:25.534767 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.633765 29538 iamrole.go:89] ListRoles: { } I0908 14:18:25.635348 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:18:25.635619 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.635746 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.636031 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0908 14:18:25.636193 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:25.636649 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0908 14:18:25.636648 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.636823 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0908 14:18:25.636932 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.636973 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0908 14:18:25.637079 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0908 14:18:25.637132 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:25.637156 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0908 14:18:25.637218 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:25.637258 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:25.637347 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.637393 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:25.637541 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:25.637596 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0908 14:18:25.637794 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0908 14:18:25.638037 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:25.638211 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:25.638275 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.638406 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:25.638931 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:25.639277 29538 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0908 14:18:25.639227 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.639351 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0908 14:18:25.639464 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0908 14:18:25.639514 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.639469 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0908 14:18:25.639689 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0908 14:18:25.639689 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0908 14:18:25.639878 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0908 14:18:25.639933 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:25.640048 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:25.640103 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:25.640151 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.640335 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.640343 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0908 14:18:25.640422 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:25.640498 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.640687 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:25.640769 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.640880 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0908 14:18:25.640985 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:25.641024 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:25.641085 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:25.641127 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:25.641267 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:25.641287 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:25.641411 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:25.641474 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:25.641524 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0908 14:18:25.641338 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0908 14:18:25.644471 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:25.644711 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.644955 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.645141 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.645256 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.645480 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.645778 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.645891 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:25.646061 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.646137 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.646825 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.646995 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.647073 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:25.647264 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0908 14:18:25.647679 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.647963 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.648233 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.648519 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:25.648738 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:25.648995 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I0908 14:18:25.651140 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.651387 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0908 14:18:25.651544 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0908 14:18:25.651685 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:25.651734 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:25.651780 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:25.652007 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:25.652334 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:25.652388 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.652426 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:25.652489 29538 dns.go:238] Pre-creating DNS records I0908 14:18:25.652527 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:25.658527 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:25.661278 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.661370 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0908 14:18:25.701771 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.798378 29538 iamrole.go:89] ListRoles: { } I0908 14:18:25.799748 29538 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0908 14:18:25.799986 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.800257 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0908 14:18:25.801507 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.801630 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:25.801798 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.802059 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:25.803561 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.804089 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:25.804216 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:25.804307 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.807090 29538 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0908 14:18:25.807225 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0908 14:18:25.807301 29538 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0908 14:18:25.807313 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0908 14:18:25.807463 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.807625 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:25.807787 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.808039 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.808287 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:25.808358 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0908 14:18:25.808605 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:25.808657 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0908 14:18:25.808797 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:25.813038 29538 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0908 14:18:25.813182 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.813388 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.813531 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.813745 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.813992 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.814157 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.814309 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:25.816194 29538 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0908 14:18:25.816476 29538 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0908 14:18:25.819850 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:25.819919 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:25.819962 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:25.819997 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:25.820035 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:25.820069 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:25.820101 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:25.820134 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:25.820171 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:25.820208 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:25.820246 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:25.820290 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:25.820324 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:25.820357 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:25.820395 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:25.820428 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:25.820463 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:25.820500 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:25.820544 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:25.820620 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:25.820691 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:25.820726 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:25.820764 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:25.820800 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:25.820832 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:25.822204 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:25.822236 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:25.822297 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822353 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0908 14:18:25.822408 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822470 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822516 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822592 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822645 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822685 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0908 14:18:25.822733 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822782 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0908 14:18:25.822822 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0908 14:18:25.822869 29538 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0908 14:18:25.822904 29538 api.go:53] PageSize not implemented I0908 14:18:25.822926 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:25.822959 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:25.822978 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:25.823022 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:25.823042 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:25.823097 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:25.823126 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:25.823177 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:25.823220 29538 iamrole.go:89] ListRoles: { } I0908 14:18:25.823295 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:25.823344 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:25.823378 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:25.823974 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0908 14:18:25.824053 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0908 14:18:25.824036 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I0908 14:18:25.824009 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0908 14:18:25.824202 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0908 14:18:25.824135 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0908 14:18:25.824291 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok W0908 14:18:25.824314 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:25.824335 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:25.824390 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:25.824480 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0908 14:18:25.824567 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:25.824621 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0908 14:18:25.824810 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:25.824831 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:25.824865 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0908 14:18:25.824930 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0908 14:18:25.824975 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:25.825069 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0908 14:18:25.825127 29538 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0908 14:18:25.825143 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0908 14:18:25.825130 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:25.825183 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0908 14:18:25.825238 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0908 14:18:25.825296 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0908 14:18:25.825313 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0908 14:18:25.825345 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0908 14:18:25.825418 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:25.825498 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:25.825574 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:25.825854 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.35s) === RUN TestLifecycleMinimalOpenstack I0908 14:18:25.826137 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0908 14:18:25.835799 29538 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:25.908064 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0908 14:18:26.060930 29538 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0908 14:18:26.060988 29538 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0908 14:18:26.077149 29538 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0908 14:18:26.087372 29538 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0908 14:18:26.152402 29538 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0908 14:18:26.156081 29538 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0908 14:18:26.159432 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:26.159530 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:26.159442 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0908 14:18:26.164992 29538 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:26.214292 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0908 14:18:26.363369 29538 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0908 14:18:26.363419 29538 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0908 14:18:26.382389 29538 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0908 14:18:26.391954 29538 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0908 14:18:26.403791 29538 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0908 14:18:26.406601 29538 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0908 14:18:26.409858 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:26.410584 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:26.411028 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0908 14:18:26.412218 29538 executor.go:111] Tasks: 96 done / 96 total; 0 can run Instance:0d880d64-555b-4b0e-b235-568f632237f7 ok SecurityGroup:f2c7dba8-0833-4cd7-89e3-cb433d523c54 ok Network:2c90df74-83c0-4778-89fa-4dcfd8d94079 ok SecurityGroup:333a3e81-78a3-458d-94fd-2f1cafd5bcd8 ok Port:649c4ac4-e3e0-457d-a3fd-937abf95cd13 ok Instance:1fe2a924-59f8-4531-afb5-4be3e43c9a0a ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Subnet:184ba2e0-b6d6-4258-a34b-e6cd7502b982 ok Port:9912b0d2-3a63-4b8d-85ba-8926e8f818f9 ok Instance:c672ba2a-6cce-41e5-9731-9c45a84b968f ok ServerGroup:55f2dde4-16e1-49d4-bbff-deb34884d01a ok ServerGroup:31efddb2-297d-4c1c-b0d1-588ca7784eeb ok Port:558171dc-8dc2-4cd9-883c-7c46422a13c0 ok SecurityGroup:e4deae45-71dd-4543-800d-02099a22c4a4 ok --- PASS: TestLifecycleMinimalOpenstack (1.69s) === RUN TestLifecycleMinimalGCE I0908 14:18:27.520086 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0908 14:18:27.522132 29538 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0908 14:18:27.760765 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:18:27.760861 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:18:27.760979 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0908 14:18:27.761015 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0908 14:18:27.761041 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:18:27.761074 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:18:27.761149 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:18:27.763362 29538 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0908 14:18:27.766696 29538 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0908 14:18:27.766774 29538 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0908 14:18:27.776376 29538 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0908 14:18:27.778964 29538 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0908 14:18:27.779194 29538 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0908 14:18:27.779294 29538 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0908 14:18:27.986570 29538 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0908 14:18:27.986674 29538 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0908 14:18:27.986760 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0908 14:18:27.986790 29538 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0908 14:18:27.986811 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:18:27.986837 29538 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0908 14:18:27.986897 29538 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0908 14:18:27.988317 29538 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0908 14:18:27.995790 29538 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0908 14:18:28.001146 29538 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0908 14:18:28.003599 29538 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0908 14:18:28.003750 29538 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0908 14:18:28.005845 29538 gce.go:98] Scanning zones: [us-test1-a] I0908 14:18:28.006219 29538 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0908 14:18:28.006245 29538 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:master-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-events-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1662646707 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1662646707 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.49s) === RUN TestLifecycleFloatingIPOpenstack I0908 14:18:28.007719 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0908 14:18:28.016434 29538 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:28.085416 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0908 14:18:28.231955 29538 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0908 14:18:28.232002 29538 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0908 14:18:28.247741 29538 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0908 14:18:28.257493 29538 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0908 14:18:28.268415 29538 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0908 14:18:28.272410 29538 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0908 14:18:28.274830 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:28.275037 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:28.275253 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0908 14:18:28.279793 29538 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:28.327525 29538 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0908 14:18:28.480578 29538 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0908 14:18:28.480643 29538 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0908 14:18:28.500215 29538 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0908 14:18:28.507022 29538 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0908 14:18:28.517533 29538 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0908 14:18:28.520161 29538 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0908 14:18:28.523921 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:28.525887 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0908 14:18:28.527406 29538 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0908 14:18:28.529873 29538 executor.go:111] Tasks: 98 done / 98 total; 0 can run Port:43e9eea9-0656-4c95-9478-c63e52842f68 ok Network:e06158a4-b8fe-4385-be6d-48698e07118b ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Router-IF:0bd65f59-1352-4e8b-bbc4-7df169869bc5 ok Router:86f46600-4d6b-4144-9f8e-c83b98b59bd3 ok ServerGroup:ccbb6b27-7447-4be6-9fac-619760d75990 ok Port:d9cbbdab-0608-4ef5-9651-16b1cc88fd06 ok SecurityGroup:92f94df4-a9a8-47e0-9a03-2fe9176c61e4 ok SecurityGroup:019c2506-9abd-4b20-a326-1139568a2d15 ok Instance:c9bc06b8-60a0-4567-9714-dac23609bc68 ok Port:6a21b4b5-2b81-4fc9-94b5-b5e1b36d5547 ok Instance:90b09766-aa76-4be8-a9ab-d688bacb0d64 ok Subnet:0bd65f59-1352-4e8b-bbc4-7df169869bc5 ok SecurityGroup:7fa41533-70b1-4e5c-aa8d-30e6a6cf68e0 ok ServerGroup:abccb15b-16f9-4d58-bd80-f40e982c83db ok Port:e68e7e3c-6115-4939-b58d-ea9f1f5a5d00 ok Instance:3174ca20-7eff-4cd4-b56b-17d0a0059622 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.63s) === RUN TestLifecyclePrivateCalico I0908 14:18:29.640372 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:29.640667 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:29.640727 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:29.640772 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:29.640823 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:29.640865 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:29.640895 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:29.640939 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:29.640980 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:29.641043 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0908 14:18:29.670257 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:29.673601 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:29.673697 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:29.673763 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:29.716109 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.138390 29538 iamrole.go:89] ListRoles: { } I0908 14:18:30.140736 29538 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0908 14:18:30.141128 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.141228 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.141311 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:30.141396 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.141613 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0908 14:18:30.141622 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.141745 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0908 14:18:30.143705 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0908 14:18:30.143829 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0908 14:18:30.141879 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:30.144292 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:30.144381 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.144376 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0908 14:18:30.144537 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0908 14:18:30.144655 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.144898 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:30.145089 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.145141 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.145491 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.146276 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:30.146756 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:30.146829 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:30.146867 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:30.146894 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:30.146942 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.148537 29538 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0908 14:18:30.148701 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.148878 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0908 14:18:30.148954 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0908 14:18:30.148933 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0908 14:18:30.149165 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0908 14:18:30.149133 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.149261 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0908 14:18:30.149336 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.149472 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:30.149543 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0908 14:18:30.149638 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0908 14:18:30.149559 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0908 14:18:30.149710 29538 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0908 14:18:30.149759 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0908 14:18:30.149822 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.149937 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:30.150027 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0908 14:18:30.150065 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0908 14:18:30.150180 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:30.150227 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0908 14:18:30.150318 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.150422 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.150502 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0908 14:18:30.150552 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0908 14:18:30.150631 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.150715 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.150807 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0908 14:18:30.150812 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:30.150911 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.151006 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:30.151039 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.151119 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0908 14:18:30.151199 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:30.151248 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:30.151341 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:30.151398 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:30.151440 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0908 14:18:30.151542 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:30.151592 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:30.151264 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0908 14:18:30.151639 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0908 14:18:30.151850 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:30.151812 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0908 14:18:30.151905 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:30.151955 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:30.152001 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0908 14:18:30.152082 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:30.152124 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:30.152174 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:30.152257 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:30.152307 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.152417 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:30.152469 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:30.152504 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:30.153844 29538 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0908 14:18:30.154075 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } W0908 14:18:30.154241 29538 api.go:53] PageSize not implemented W0908 14:18:30.154404 29538 api.go:53] PageSize not implemented I0908 14:18:30.154307 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" }] }] } I0908 14:18:30.154535 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0908 14:18:30.154550 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0908 14:18:30.154698 29538 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" }] } I0908 14:18:30.154783 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0908 14:18:30.154723 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0908 14:18:30.154832 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0908 14:18:30.154899 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0908 14:18:30.155003 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.155074 29538 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I0908 14:18:30.155132 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.155161 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0908 14:18:30.154988 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.155209 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0908 14:18:30.155254 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.155410 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0908 14:18:30.155453 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0908 14:18:30.155525 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0908 14:18:30.155729 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.155857 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.156020 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.156148 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.156270 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.156360 29538 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.156370 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" }] }] } I0908 14:18:30.156522 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.156656 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.156801 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.156937 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.157063 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0908 14:18:30.157199 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:30.157338 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.157441 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.157523 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:30.157613 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.157782 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:30.157835 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.157989 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.158125 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.158259 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.158402 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0908 14:18:30.158532 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.158670 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.158795 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0908 14:18:30.158898 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.159006 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.159084 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0908 14:18:30.159282 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0908 14:18:30.159443 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:30.159490 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:30.159530 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0908 14:18:30.159674 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.159813 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:30.159936 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0908 14:18:30.160090 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.160278 29538 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0908 14:18:30.160394 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:30.160501 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.160621 29538 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:30.160593 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:30.160777 29538 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0908 14:18:30.160881 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:30.160927 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:30.160969 29538 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0908 14:18:30.161035 29538 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0908 14:18:30.161100 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.161237 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0908 14:18:30.161305 29538 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0908 14:18:30.161347 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.161379 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:30.161425 29538 dns.go:238] Pre-creating DNS records I0908 14:18:30.161450 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:30.167930 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:30.171257 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.171348 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.171414 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.211924 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.604384 29538 iamrole.go:89] ListRoles: { } I0908 14:18:30.606770 29538 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0908 14:18:30.607054 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.607402 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:30.607412 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.607612 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.607772 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:30.607808 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:30.607862 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.608091 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.608174 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.608301 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.610081 29538 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0908 14:18:30.610242 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.610468 29538 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0908 14:18:30.610412 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.610596 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0908 14:18:30.610672 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0908 14:18:30.610586 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0908 14:18:30.610937 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:30.611037 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0908 14:18:30.611283 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:30.611377 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.611568 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0908 14:18:30.611718 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:30.611761 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.611918 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:30.611982 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.612091 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0908 14:18:30.612277 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.612421 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0908 14:18:30.612554 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0908 14:18:30.612672 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:30.612752 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:30.612824 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:30.615543 29538 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0908 14:18:30.615721 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0908 14:18:30.615813 29538 api.go:53] PageSize not implemented I0908 14:18:30.615847 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.615856 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.615996 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0908 14:18:30.615985 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.616060 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } W0908 14:18:30.616208 29538 api.go:53] PageSize not implemented I0908 14:18:30.616240 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.616340 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.616257 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.616388 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0908 14:18:30.616618 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.616750 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.616904 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.617012 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:30.617064 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0908 14:18:30.617244 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0908 14:18:30.617385 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.617570 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.617737 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.617952 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:30.620862 29538 executor.go:111] Tasks: 118 done / 124 total; 5 can run W0908 14:18:30.621048 29538 api.go:53] PageSize not implemented I0908 14:18:30.621032 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:30.621091 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.621169 29538 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0908 14:18:30.621206 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:30.621087 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } W0908 14:18:30.621355 29538 api.go:53] PageSize not implemented I0908 14:18:30.621400 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0908 14:18:30.621637 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0908 14:18:30.621778 29538 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0908 14:18:30.621857 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.621971 29538 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0908 14:18:30.625775 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:30.625837 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:30.625873 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:30.625907 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:30.625946 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0908 14:18:30.625982 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:30.626018 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:30.626051 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:30.626083 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:30.626117 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:30.626151 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:30.626193 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:30.626234 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0908 14:18:30.626270 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:30.626304 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:30.626336 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:30.626369 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:30.626404 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:30.626436 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:30.626472 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:30.626507 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:30.626542 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:30.626577 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:30.626612 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:30.626647 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:30.626683 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:30.626717 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:30.626754 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:30.626787 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:30.626819 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:30.626856 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:30.626893 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:30.626928 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:30.628683 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:30.628712 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:30.628757 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.628824 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0908 14:18:30.628904 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.628962 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629011 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629063 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629105 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629148 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0908 14:18:30.629208 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629263 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:30.629290 29538 natgateway.go:117] DescribeNatGateways: { } I0908 14:18:30.629326 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0908 14:18:30.629367 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0908 14:18:30.629422 29538 api.go:53] PageSize not implemented I0908 14:18:30.629439 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0908 14:18:30.629492 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:30.629519 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:30.629538 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:30.629562 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:30.629582 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:30.629637 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.629667 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:30.629712 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:30.629761 29538 iamrole.go:89] ListRoles: { } I0908 14:18:30.629811 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:30.629861 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:30.629904 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0908 14:18:30.629941 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:30.629978 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0908 14:18:30.630623 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-group:nodes.privatecalico.example.com ok I0908 14:18:30.630670 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0908 14:18:30.630748 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:30.630795 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0908 14:18:30.630638 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:masters.privatecalico.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I0908 14:18:30.630803 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I0908 14:18:30.630848 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:bastion.privatecalico.example.com ok I0908 14:18:30.630884 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0908 14:18:30.630973 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0908 14:18:30.630967 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0908 14:18:30.631029 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0908 14:18:30.631103 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0908 14:18:30.630943 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0908 14:18:30.631154 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:30.631172 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0908 14:18:30.631214 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0908 14:18:30.631249 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:30.631324 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:30.631388 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0908 14:18:30.631481 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:30.631519 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0908 14:18:30.631556 29538 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0908 14:18:30.631692 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:30.631782 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W0908 14:18:30.632027 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:30.632053 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:30.632091 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-3 ok I0908 14:18:30.632215 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0908 14:18:30.632232 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0908 14:18:30.632276 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0908 14:18:30.632344 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:30.632390 29538 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0908 14:18:30.632412 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0908 14:18:30.632463 29538 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0908 14:18:30.632510 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0908 14:18:30.632400 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:30.632537 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0908 14:18:30.632590 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0908 14:18:30.632638 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0908 14:18:30.632704 29538 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0908 14:18:30.632732 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0908 14:18:30.632582 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } I0908 14:18:30.632781 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok W0908 14:18:30.632799 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:30.632825 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:30.632867 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:30.632928 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } W0908 14:18:30.633034 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:30.633054 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0908 14:18:30.633089 29538 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0908 14:18:30.633200 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0908 14:18:30.633260 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:30.633335 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:30.633414 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:30.633688 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.99s) === RUN TestLifecyclePrivateKopeio I0908 14:18:30.634011 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:30.634269 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:30.634320 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:30.634360 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:30.634406 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:30.634448 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:30.634482 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:30.634517 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:30.634546 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:30.634604 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0908 14:18:30.664965 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:30.668087 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.668183 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.668245 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.709581 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.825923 29538 iamrole.go:89] ListRoles: { } I0908 14:18:30.828509 29538 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0908 14:18:30.828721 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.828868 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.828936 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0908 14:18:30.829050 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] } I0908 14:18:30.829074 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.829129 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:30.829195 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:30.829230 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0908 14:18:30.829283 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:30.829358 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0908 14:18:30.829436 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:30.829484 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.829555 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0908 14:18:30.829624 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0908 14:18:30.829644 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.829854 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.829861 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.829922 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:30.830046 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.830200 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:30.830234 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:30.830256 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:30.830297 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.830328 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:30.830423 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:30.831846 29538 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0908 14:18:30.832110 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0908 14:18:30.832087 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.832277 29538 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0908 14:18:30.832319 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0908 14:18:30.832259 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0908 14:18:30.832418 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0908 14:18:30.832388 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0908 14:18:30.832520 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.832626 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0908 14:18:30.832782 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0908 14:18:30.832770 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0908 14:18:30.832870 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0908 14:18:30.832757 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.833032 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:30.833118 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0908 14:18:30.833198 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0908 14:18:30.833353 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0908 14:18:30.833571 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.833683 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.833789 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0908 14:18:30.833914 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.834085 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.834221 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:30.834202 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0908 14:18:30.834306 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.834399 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0908 14:18:30.834464 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.834609 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:30.834720 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0908 14:18:30.834527 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0908 14:18:30.834849 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.835027 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.835127 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:30.835176 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:30.835234 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.835327 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0908 14:18:30.835379 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:30.835338 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:30.835428 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:30.835489 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:30.835566 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:30.835627 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0908 14:18:30.835705 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.835800 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0908 14:18:30.835865 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0908 14:18:30.835923 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:30.836015 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:30.836068 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:30.835981 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:30.836110 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:30.836048 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:30.836149 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:30.836209 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:30.836253 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:30.836312 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:30.836366 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:30.836454 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0908 14:18:30.836509 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:30.836571 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:30.836617 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0908 14:18:30.836710 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:30.836768 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0908 14:18:30.836811 29538 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0908 14:18:30.836892 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:30.836941 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:30.836984 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:30.837033 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0908 14:18:30.837387 29538 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0908 14:18:30.837496 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0908 14:18:30.837557 29538 api.go:53] PageSize not implemented W0908 14:18:30.837624 29538 api.go:53] PageSize not implemented I0908 14:18:30.837617 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0908 14:18:30.837715 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.837820 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:30.837776 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0908 14:18:30.837879 29538 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0908 14:18:30.837929 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:30.837905 29538 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" }] } I0908 14:18:30.838000 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0908 14:18:30.837979 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.838067 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:30.838150 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:30.838192 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0908 14:18:30.838174 29538 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I0908 14:18:30.838241 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:30.838274 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0908 14:18:30.838332 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0908 14:18:30.838326 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0908 14:18:30.838406 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0908 14:18:30.838457 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0908 14:18:30.838508 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0908 14:18:30.838565 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0908 14:18:30.838613 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.838704 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:30.838874 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.838966 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:30.839059 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0908 14:18:30.839140 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0908 14:18:30.839184 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.839343 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0908 14:18:30.839458 29538 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0908 14:18:30.839508 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.839641 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.839784 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.839916 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.840044 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:30.840091 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.840237 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.840331 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0908 14:18:30.840374 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.840465 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.840638 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.840780 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:30.840894 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0908 14:18:30.840993 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.841129 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:30.841240 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:30.841289 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.841432 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.841562 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-4" } I0908 14:18:30.841598 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0908 14:18:30.841713 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.841836 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.841961 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0908 14:18:30.842093 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:30.842214 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.842345 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:30.842392 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0908 14:18:30.842529 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.842615 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0908 14:18:30.842663 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.842711 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0908 14:18:30.842745 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.842791 29538 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0908 14:18:30.842842 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0908 14:18:30.842890 29538 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0908 14:18:30.842930 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:30.842970 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0908 14:18:30.843019 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.843139 29538 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0908 14:18:30.845730 29538 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0908 14:18:30.845848 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0908 14:18:30.845933 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:30.845994 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:30.846132 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I0908 14:18:30.846108 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:30.846192 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0908 14:18:30.846258 29538 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0908 14:18:30.846301 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:30.846333 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:30.846384 29538 dns.go:238] Pre-creating DNS records I0908 14:18:30.846414 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:30.855166 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:30.860412 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.860518 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.860590 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:30.900540 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.005721 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.007712 29538 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0908 14:18:31.007946 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:31.008086 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.008257 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.008346 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.008467 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.011660 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.011975 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:31.012075 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.012551 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:31.012694 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.015153 29538 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0908 14:18:31.015312 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0908 14:18:31.015427 29538 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0908 14:18:31.015514 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0908 14:18:31.015716 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0908 14:18:31.015709 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.015907 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0908 14:18:31.015918 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:31.015964 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.016103 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:31.016172 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.016268 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0908 14:18:31.016438 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:31.016500 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.016721 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:31.016769 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.016909 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:31.016949 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.017066 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.017177 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0908 14:18:31.017334 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.017449 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.017519 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0908 14:18:31.017633 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.017756 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:31.017796 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.017846 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0908 14:18:31.017884 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.020693 29538 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0908 14:18:31.020831 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0908 14:18:31.021041 29538 api.go:53] PageSize not implemented I0908 14:18:31.021031 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0908 14:18:31.021079 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:31.021188 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:31.021228 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0908 14:18:31.021196 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0908 14:18:31.021318 29538 api.go:53] PageSize not implemented I0908 14:18:31.021342 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:31.021336 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:31.021411 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0908 14:18:31.021448 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0908 14:18:31.021548 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:31.021755 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.021940 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:31.022078 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:31.022127 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.022210 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0908 14:18:31.022270 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0908 14:18:31.022361 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.022434 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.022558 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:31.025127 29538 executor.go:111] Tasks: 117 done / 123 total; 6 can run W0908 14:18:31.025223 29538 api.go:53] PageSize not implemented I0908 14:18:31.025243 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0908 14:18:31.025250 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0908 14:18:31.025413 29538 api.go:53] PageSize not implemented I0908 14:18:31.025362 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.025458 29538 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0908 14:18:31.025368 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.025435 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:31.025625 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0908 14:18:31.025773 29538 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0908 14:18:31.029531 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:31.029597 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:31.029634 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:31.029668 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:31.029710 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:31.029746 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:31.029779 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:31.029815 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:31.029850 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:31.029886 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:31.029924 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:31.029963 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0908 14:18:31.030016 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:31.030055 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:31.030090 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:31.030126 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:31.030159 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:31.030196 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0908 14:18:31.030233 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:31.030273 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:31.030310 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:31.030348 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:31.030386 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:31.030423 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.030462 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:31.030498 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:31.030539 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0908 14:18:31.030578 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0908 14:18:31.030638 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:31.030676 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:31.030711 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:31.030750 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:31.030787 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:31.030825 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:31.032617 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:31.032649 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:31.032692 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.032758 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0908 14:18:31.032839 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.032903 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.032955 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.033012 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.033055 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.033108 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0908 14:18:31.033185 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.033257 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:31.033293 29538 natgateway.go:117] DescribeNatGateways: { } I0908 14:18:31.033345 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0908 14:18:31.033390 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0908 14:18:31.033453 29538 api.go:53] PageSize not implemented I0908 14:18:31.033472 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0908 14:18:31.033528 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:31.033557 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:31.033575 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:31.033601 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:31.033620 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:31.033674 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.033705 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.033758 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:31.033797 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.033842 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:31.033893 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:31.033965 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1","rtb-3"] } I0908 14:18:31.034016 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0908 14:18:31.034052 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } autoscaling-group:nodes.privatekopeio.example.com ok I0908 14:18:31.034717 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-config:lt-1 ok I0908 14:18:31.034818 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0908 14:18:31.034807 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0908 14:18:31.034884 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0908 14:18:31.034798 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0908 14:18:31.034917 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0908 14:18:31.034967 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0908 14:18:31.035012 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } volume:vol-1 ok I0908 14:18:31.035032 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0908 14:18:31.035096 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0908 14:18:31.034910 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.035094 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok autoscaling-group:bastion.privatekopeio.example.com ok I0908 14:18:31.035132 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0908 14:18:31.035181 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0908 14:18:31.035159 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0908 14:18:31.035258 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.035274 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0908 14:18:31.035313 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0908 14:18:31.035520 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.035542 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0908 14:18:31.035589 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0908 14:18:31.035666 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:31.035716 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0908 14:18:31.035826 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.035845 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0908 14:18:31.035878 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0908 14:18:31.036000 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.036040 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0908 14:18:31.036055 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0908 14:18:31.036125 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com W0908 14:18:31.036136 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.036151 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:31.036143 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0908 14:18:31.036198 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0908 14:18:31.036196 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.036239 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0908 14:18:31.036290 29538 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0908 14:18:31.036309 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0908 14:18:31.036343 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0908 14:18:31.036292 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0908 14:18:31.036417 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.036375 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0908 14:18:31.036437 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0908 14:18:31.036452 29538 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com security-group:sg-1 ok I0908 14:18:31.036470 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0908 14:18:31.036478 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0908 14:18:31.036508 29538 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } subnet:subnet-1 ok iam-role:bastions.privatekopeio.example.com ok I0908 14:18:31.036544 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0908 14:18:31.036611 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0908 14:18:31.036658 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0908 14:18:31.036757 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0908 14:18:31.036815 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:31.036869 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0908 14:18:31.036946 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:31.037019 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:31.037316 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.40s) === RUN TestLifecycleIPv6 I0908 14:18:31.037625 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:31.037895 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:31.037943 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:31.037987 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:31.038034 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:31.038074 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:31.038106 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:31.038140 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:31.038165 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:31.038218 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0908 14:18:31.063685 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:31.066973 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.067065 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.107093 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.279232 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.280852 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:18:31.281286 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.281366 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.281381 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.281406 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:31.281460 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0908 14:18:31.281565 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.281736 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.281867 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0908 14:18:31.281977 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0908 14:18:31.283994 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I0908 14:18:31.284742 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.285176 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.285537 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0908 14:18:31.287014 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:31.287141 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:31.287393 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:31.288206 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:31.288347 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:31.285806 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.288522 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:31.288564 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:31.288603 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:31.288651 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.289691 29538 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0908 14:18:31.289828 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.289905 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0908 14:18:31.289955 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.290038 29538 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0908 14:18:31.289989 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0908 14:18:31.290133 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.290107 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0908 14:18:31.290311 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0908 14:18:31.290281 29538 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0908 14:18:31.290409 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:31.290486 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.290585 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:31.290666 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.290816 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:31.290865 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0908 14:18:31.291065 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:31.291172 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:31.291221 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.291127 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0908 14:18:31.291259 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.291359 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.291436 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0908 14:18:31.291582 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:31.291673 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:31.291733 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:31.291794 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:31.291881 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:31.291930 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0908 14:18:31.292020 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:31.292162 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:31.292226 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:31.292272 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.292318 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0908 14:18:31.292469 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:31.292568 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:31.292615 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0908 14:18:31.292482 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.295643 29538 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0908 14:18:31.295769 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.295943 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:31.296011 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.296176 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0908 14:18:31.296236 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.296499 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I0908 14:18:31.296736 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:31.296960 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.297175 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.297358 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:31.297511 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.297533 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0908 14:18:31.297588 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.297757 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.297803 29538 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0908 14:18:31.297903 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.298033 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.298237 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0908 14:18:31.298375 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.298588 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.298664 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:31.298733 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0908 14:18:31.298936 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0908 14:18:31.299109 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:31.299197 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0908 14:18:31.299363 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0908 14:18:31.299537 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0908 14:18:31.299711 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.299846 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.299955 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0908 14:18:31.300122 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0908 14:18:31.300195 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0908 14:18:31.300319 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.300477 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.300635 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.300791 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.300925 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.301027 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.303621 29538 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0908 14:18:31.303732 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.303784 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:31.303877 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:31.303834 29538 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.303975 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.304025 29538 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0908 14:18:31.304035 29538 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0908 14:18:31.304155 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:31.304171 29538 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0908 14:18:31.304216 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0908 14:18:31.304257 29538 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0908 14:18:31.304276 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:31.304356 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.304409 29538 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0908 14:18:31.304475 29538 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0908 14:18:31.304543 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.304646 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.304680 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0908 14:18:31.304820 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.304950 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.305077 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.305182 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:18:31.305224 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.305257 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.305306 29538 dns.go:238] Pre-creating DNS records I0908 14:18:31.305332 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.311318 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:31.314651 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.314750 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.355274 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.528708 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.530583 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:18:31.530795 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I0908 14:18:31.531109 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.531341 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.531533 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.531457 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:31.531640 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.531823 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.531942 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:31.531996 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:31.532084 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.533745 29538 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0908 14:18:31.533906 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.534129 29538 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0908 14:18:31.534140 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0908 14:18:31.534251 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0908 14:18:31.534277 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0908 14:18:31.534153 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.534429 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0908 14:18:31.534613 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.534683 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0908 14:18:31.534873 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.534971 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0908 14:18:31.535247 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:31.535304 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:31.535404 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.535553 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.539162 29538 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0908 14:18:31.539382 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.539529 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.539716 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.539891 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.540102 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.540188 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:31.540231 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0908 14:18:31.540271 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.540411 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.540583 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.540821 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0908 14:18:31.541022 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:31.541083 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.541172 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0908 14:18:31.543078 29538 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0908 14:18:31.543162 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.543201 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:31.543319 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:31.543297 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.543366 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:31.543352 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0908 14:18:31.543470 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0908 14:18:31.543515 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0908 14:18:31.543562 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0908 14:18:31.543602 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0908 14:18:31.543708 29538 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0908 14:18:31.543779 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.543859 29538 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0908 14:18:31.543885 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:31.543909 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0908 14:18:31.543883 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:31.543980 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:31.544098 29538 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0908 14:18:31.544129 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:31.544156 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0908 14:18:31.544247 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:18:31.547387 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:31.547449 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:31.547487 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:31.547519 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:31.547558 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0908 14:18:31.547594 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:31.547627 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:31.547660 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:31.547693 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:31.547728 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:31.547763 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:31.547803 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:31.547845 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:31.547884 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:31.547917 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:31.547950 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:31.547986 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:31.548018 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:31.548053 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:31.548088 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:31.548122 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:31.548157 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.548192 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:31.548224 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:31.548256 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:31.548288 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:31.548326 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:31.548363 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:31.548399 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:31.549967 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:31.549996 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:31.550041 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550094 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0908 14:18:31.550157 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550221 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550275 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550326 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550366 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550409 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0908 14:18:31.550462 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550507 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:31.550534 29538 natgateway.go:117] DescribeNatGateways: { } I0908 14:18:31.550575 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0908 14:18:31.550635 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0908 14:18:31.550694 29538 api.go:53] PageSize not implemented I0908 14:18:31.550715 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:31.550742 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:31.550757 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0908 14:18:31.550799 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:31.550822 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:31.550840 29538 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0908 14:18:31.550906 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.550934 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.550978 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:31.551015 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.551061 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:31.551116 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:31.551163 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0908 14:18:31.551201 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:31.551235 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal-ipv6.example.com ok I0908 14:18:31.551818 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:31.551887 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0908 14:18:31.551886 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.551925 29538 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0908 14:18:31.551956 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.551857 29538 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0908 14:18:31.551982 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } nat-gateway:nat-1 ok I0908 14:18:31.551996 29538 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0908 14:18:31.552032 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0908 14:18:31.551860 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0908 14:18:31.552093 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } I0908 14:18:31.552059 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } internet-gateway:igw-2 ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok volume:vol-1 ok I0908 14:18:31.552151 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal-ipv6.example.com ok I0908 14:18:31.552154 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:nodes.minimal-ipv6.example.com ok volume:vol-2 ok I0908 14:18:31.552212 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.552268 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0908 14:18:31.552561 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.552586 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:31.552623 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0908 14:18:31.552686 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.552742 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0908 14:18:31.552874 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.552896 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0908 14:18:31.552940 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:31.553042 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.553103 29538 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0908 14:18:31.553073 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:31.553119 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0908 14:18:31.553174 29538 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0908 14:18:31.553220 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0908 14:18:31.553282 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0908 14:18:31.553341 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0908 14:18:31.553357 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0908 14:18:31.553388 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0908 14:18:31.553461 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0908 14:18:31.553546 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:31.553639 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:31.553925 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.52s) === RUN TestLifecycleSharedVPC I0908 14:18:31.554220 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:31.554510 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:31.554555 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:31.554596 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:31.554647 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:31.554696 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:31.554729 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:31.554765 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:31.554792 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:31.554844 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0908 14:18:31.584083 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:31.587214 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.587337 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.626990 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.727032 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.728747 29538 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0908 14:18:31.729006 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0908 14:18:31.729181 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0908 14:18:31.729235 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:31.730452 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.730928 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0908 14:18:31.730970 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:31.731084 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0908 14:18:31.731201 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:31.731284 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I0908 14:18:31.731510 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:31.731753 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:31.731848 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.731963 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.732277 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.732860 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:31.733352 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:31.734701 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:31.737371 29538 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0908 14:18:31.737560 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0908 14:18:31.737680 29538 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0908 14:18:31.737599 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0908 14:18:31.737821 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0908 14:18:31.737902 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.738029 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0908 14:18:31.738172 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0908 14:18:31.738237 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0908 14:18:31.738298 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0908 14:18:31.738381 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:31.738527 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:31.738582 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.738638 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.738690 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:31.738745 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.738863 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:31.738953 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:31.739121 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:31.739052 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0908 14:18:31.739193 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:31.739347 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.739431 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:31.739509 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0908 14:18:31.740218 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0908 14:18:31.742926 29538 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0908 14:18:31.743147 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.743345 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.743586 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0908 14:18:31.743760 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:31.744084 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" }] }] } I0908 14:18:31.744319 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.744456 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.744540 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.744622 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:31.744783 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.744939 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.745086 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.745190 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.745341 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0908 14:18:31.745479 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0908 14:18:31.745535 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:31.745582 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:31.745623 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.745726 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.745798 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" }] }] } I0908 14:18:31.745994 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.746140 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0908 14:18:31.746288 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0908 14:18:31.746430 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0908 14:18:31.746577 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.748076 29538 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0908 14:18:31.748344 29538 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0908 14:18:31.748392 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.748442 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.748495 29538 dns.go:238] Pre-creating DNS records I0908 14:18:31.748526 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.754274 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:31.757127 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.757218 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.798089 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.892788 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.894476 29538 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0908 14:18:31.894911 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:31.895076 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:31.895050 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:31.895146 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:31.895231 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.895457 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.895689 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.895762 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:31.897975 29538 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0908 14:18:31.898177 29538 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0908 14:18:31.898269 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0908 14:18:31.898115 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0908 14:18:31.898344 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.898596 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0908 14:18:31.898850 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.898941 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:31.899047 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:31.899177 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I0908 14:18:31.899317 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:31.903415 29538 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0908 14:18:31.903701 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.903895 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.904274 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.904400 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.904543 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.904690 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.904766 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.906117 29538 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0908 14:18:31.906393 29538 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0908 14:18:31.909211 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:31.909276 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:31.909311 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:31.909342 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:31.909378 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:31.909411 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:31.909442 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:31.909480 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:31.909519 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:31.909562 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:31.909596 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:31.909627 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:31.909662 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:31.909692 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:31.909727 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:31.909760 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:31.909797 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:31.909829 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:31.909861 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:31.909893 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:31.909930 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:31.909966 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:31.911438 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:31.911470 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:31.911512 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911565 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0908 14:18:31.911619 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911678 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911720 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911762 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911803 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911842 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0908 14:18:31.911891 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911946 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0908 14:18:31.911981 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0908 14:18:31.912030 29538 api.go:53] PageSize not implemented I0908 14:18:31.912052 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:31.912080 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:31.912099 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:31.912124 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:31.912141 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:31.912200 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:31.912232 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:31.912278 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:31.912321 29538 iamrole.go:89] ListRoles: { } I0908 14:18:31.912363 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:31.912402 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:31.912432 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:31.912807 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.sharedvpc.example.com ok keypair:key-1 ok I0908 14:18:31.912872 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0908 14:18:31.912912 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0908 14:18:31.912997 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0908 14:18:31.913046 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0908 14:18:31.913161 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0908 14:18:31.912933 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0908 14:18:31.913215 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.912987 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok I0908 14:18:31.913234 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:31.913317 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:31.913409 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0908 14:18:31.913561 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:31.913596 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:31.913634 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0908 14:18:31.913748 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:31.913857 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:31.913898 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0908 14:18:31.914014 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0908 14:18:31.914033 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0908 14:18:31.914069 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0908 14:18:31.914106 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0908 14:18:31.914153 29538 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0908 14:18:31.914165 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0908 14:18:31.914196 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0908 14:18:31.914253 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:31.914538 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.36s) === RUN TestLifecycleComplex I0908 14:18:31.914832 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:31.915084 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:31.915131 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:31.915172 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:31.915220 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:31.915265 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:31.915296 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:31.915334 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:31.915360 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:31.915413 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0908 14:18:31.954652 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:31.959318 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.959430 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:31.998658 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.194607 29538 iamrole.go:89] ListRoles: { } I0908 14:18:32.196892 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0908 14:18:32.197084 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:32.197611 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0908 14:18:32.197707 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } W0908 14:18:32.197793 29538 api.go:53] PageSize not implemented I0908 14:18:32.197664 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.197784 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:32.197866 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0908 14:18:32.197937 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0908 14:18:32.197961 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:32.198159 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:32.198415 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:32.198493 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.198720 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:32.198961 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.198953 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.199240 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:32.199290 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:32.199347 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:32.199401 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.199445 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:32.199587 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:32.199637 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:32.199751 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0908 14:18:32.199876 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:32.199923 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:32.200064 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:18:32.200320 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:32.200266 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:32.200381 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:32.200385 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:32.200426 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.200600 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0908 14:18:32.200577 29538 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0908 14:18:32.200719 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0908 14:18:32.200689 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.200748 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:32.200816 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:32.200869 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.200901 29538 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0908 14:18:32.201051 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:32.201097 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.201226 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0908 14:18:32.201273 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0908 14:18:32.201335 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.201549 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:32.201752 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:32.201865 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.201967 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0908 14:18:32.202076 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.202158 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.202249 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.202340 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.202413 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:32.202525 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.202478 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:32.202627 29538 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0908 14:18:32.202671 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:32.202732 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:32.202773 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:32.202917 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:32.202901 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0908 14:18:32.203008 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:32.202802 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:32.203062 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0908 14:18:32.203165 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0908 14:18:32.203218 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:32.203258 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0908 14:18:32.202891 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0908 14:18:32.203410 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:32.203393 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] }], VpcId: "vpc-1" } I0908 14:18:32.203485 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:32.203568 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:32.203623 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:32.203664 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:32.203710 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:32.203827 29538 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0908 14:18:32.203880 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:32.203933 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0908 14:18:32.204057 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:32.204107 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:32.204158 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:32.205866 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:18:32.206013 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.206177 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.206193 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:32.206285 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:32.206277 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:32.206356 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:32.206465 29538 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0908 14:18:32.206526 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0908 14:18:32.206617 29538 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0908 14:18:32.206664 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.206771 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.206754 29538 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0908 14:18:32.206863 29538 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.206921 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0908 14:18:32.207073 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.207220 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:32.207324 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I0908 14:18:32.208409 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.208676 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.208985 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0908 14:18:32.209320 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.209712 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.210047 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0908 14:18:32.210268 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.210658 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0908 14:18:32.210860 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" } ] }] } I0908 14:18:32.211902 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.212090 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0908 14:18:32.213080 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0908 14:18:32.213352 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0908 14:18:32.213521 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0908 14:18:32.213634 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0908 14:18:32.213802 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.213984 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0908 14:18:32.214101 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0908 14:18:32.214206 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.214357 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0908 14:18:32.214507 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0908 14:18:32.214604 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0908 14:18:32.214751 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0908 14:18:32.214902 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.215049 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0908 14:18:32.215106 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.215259 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:32.215302 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0908 14:18:32.215474 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0908 14:18:32.215581 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0908 14:18:32.215617 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.215711 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0908 14:18:32.215863 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.216014 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0908 14:18:32.216163 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0908 14:18:32.216269 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.216415 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0908 14:18:32.219064 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:18:32.219154 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:32.219304 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:32.219419 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0908 14:18:32.219461 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.219490 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:32.219535 29538 dns.go:238] Pre-creating DNS records I0908 14:18:32.219560 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:32.228352 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:32.232644 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.232743 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.272691 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.461992 29538 iamrole.go:89] ListRoles: { } I0908 14:18:32.464276 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run W0908 14:18:32.464605 29538 api.go:53] PageSize not implemented I0908 14:18:32.464654 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.464873 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.465038 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:32.465154 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.465241 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:32.465434 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.465613 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:32.465663 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:32.465764 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.465849 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:32.466026 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:32.466199 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:32.467242 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:18:32.467398 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.467459 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:32.467554 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:32.467516 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.467670 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:32.467732 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:32.467639 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:32.467697 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:32.467807 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:32.467796 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.468057 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.468282 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.468482 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:32.468551 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.468656 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.468744 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:32.468909 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:32.468965 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.469058 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.469195 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:32.469248 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.469438 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:32.469486 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:32.469551 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:32.469705 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:32.472892 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:18:32.473044 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.473265 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:32.473380 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:32.473290 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.473399 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.473490 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.473532 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.473541 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.473573 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:32.473610 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:32.473615 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.473644 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.474015 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.474423 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.474586 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.474777 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.474928 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.475240 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.475504 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.477356 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:18:32.477461 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:32.477589 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:32.477630 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:32.477674 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.477771 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0908 14:18:32.489789 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.493498 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.505551 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:32.509496 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.509593 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.550138 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.745006 29538 iamrole.go:89] ListRoles: { } I0908 14:18:32.746828 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0908 14:18:32.747139 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:32.747185 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:32.747299 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0908 14:18:32.748675 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:32.748942 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:32.749131 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:32.749323 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.749584 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.749775 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:32.749826 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:32.749957 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0908 14:18:32.750581 29538 api.go:53] PageSize not implemented I0908 14:18:32.751253 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.754760 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:18:32.754924 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:32.755024 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:32.755075 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:32.754916 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.755108 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:32.755183 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:32.755259 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:32.755195 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.755355 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.755481 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:32.755677 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:32.755752 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.755927 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.756110 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:32.756170 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.756295 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.756429 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:32.756577 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:32.756710 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:32.756837 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:32.756995 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0908 14:18:32.757100 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:32.757143 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:32.757223 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:32.757265 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:32.760740 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:18:32.760902 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:32.760976 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:32.760935 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.761002 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.761066 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.761133 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.761202 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:32.761046 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.761271 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:32.761712 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.761751 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.761954 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0908 14:18:32.761955 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.761997 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:32.762030 29538 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0908 14:18:32.762062 29538 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0908 14:18:32.762093 29538 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0908 14:18:32.762181 29538 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0908 14:18:32.762194 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.762249 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.762331 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.762534 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.762791 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:32.763127 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.763250 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.763334 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:32.763425 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.764636 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:18:32.764814 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:32.764915 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:32.764939 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:32.764962 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:32.765049 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0908 14:18:32.765092 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:32.765121 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:32.765167 29538 dns.go:238] Pre-creating DNS records I0908 14:18:32.772976 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:32.776983 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.777083 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:32.816400 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.006472 29538 iamrole.go:89] ListRoles: { } I0908 14:18:33.008790 29538 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0908 14:18:33.009216 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0908 14:18:33.009337 29538 api.go:53] PageSize not implemented I0908 14:18:33.009311 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:33.009416 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.009615 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:33.009803 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.009986 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:33.010096 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:33.010217 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:33.010250 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:33.010312 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0908 14:18:33.010582 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:33.010820 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.012294 29538 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0908 14:18:33.012404 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.012509 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:33.012642 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.012598 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:33.012556 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.012732 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:33.012828 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.012842 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:33.013032 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:33.013030 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:33.013088 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:33.013238 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:33.013294 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.013488 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:33.013535 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.013626 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.013765 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:33.013823 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.013983 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.014149 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:33.014193 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.014285 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.014418 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:33.014593 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:33.014638 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:33.018369 29538 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0908 14:18:33.018607 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.018781 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.018892 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0908 14:18:33.018926 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0908 14:18:33.019011 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:33.019039 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.019127 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.019192 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.019264 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.019330 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.019388 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.019446 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.019623 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.019872 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.020130 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.020371 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.020464 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.020631 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.020739 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.028100 29538 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0908 14:18:33.028257 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:33.028370 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:33.028397 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:33.028426 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.028515 29538 executor.go:111] Tasks: 120 done / 120 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0908 14:18:33.032821 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.036933 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.049335 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:33.053257 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.053360 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.092316 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.288051 29538 iamrole.go:89] ListRoles: { } I0908 14:18:33.290537 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:33.290814 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:33.291014 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } W0908 14:18:33.291129 29538 api.go:53] PageSize not implemented I0908 14:18:33.291046 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:33.291185 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.291346 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:33.291569 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.291839 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:33.292021 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:33.292183 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.292369 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:33.292418 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:33.292463 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:33.292566 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.293151 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:33.293390 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:33.293525 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:33.293506 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:33.293617 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:33.293572 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.293602 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.293708 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.293777 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:33.293827 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:33.293846 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.293902 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.294078 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.294235 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:33.294281 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:33.294430 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:33.294479 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.294684 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:33.294861 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:33.294917 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.295009 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.295093 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.295173 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:33.295308 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.295444 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:33.295499 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:33.298721 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:33.298873 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.299030 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0908 14:18:33.298978 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0908 14:18:33.299106 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:33.299200 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.299263 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.299233 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.299301 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.299347 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.299383 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.299414 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.299635 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.299822 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.300023 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.300163 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.300572 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.300695 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.300821 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.300911 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.302774 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.302850 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.303071 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:33.303147 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:33.303229 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:33.303253 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:33.303278 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.303352 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0908 14:18:33.303393 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.303425 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:33.303474 29538 dns.go:238] Pre-creating DNS records I0908 14:18:33.311412 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:33.315498 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.315604 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.357248 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.550557 29538 iamrole.go:89] ListRoles: { } I0908 14:18:33.552271 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:33.552588 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0908 14:18:33.552694 29538 api.go:53] PageSize not implemented I0908 14:18:33.552761 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:33.552935 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:33.552944 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.554537 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:33.554598 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:33.554734 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.554829 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.555097 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:33.555294 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:33.555631 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.555883 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:33.556062 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:33.560198 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:33.560402 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.560547 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:33.560552 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.560680 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.560648 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:33.560727 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.560866 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:33.560761 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:33.560952 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:33.560941 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.561017 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.561172 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:33.561362 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:33.561434 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.561537 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:33.561695 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:33.561754 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.561906 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:33.561954 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.562152 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.562305 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:33.562364 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.562541 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.562688 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:33.562953 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:33.567382 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:33.567529 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:33.567621 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:33.567641 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.567687 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.567724 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.567764 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.567802 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.567834 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.567823 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.568133 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.568444 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.568669 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.568819 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.569016 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.569149 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.569288 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.569408 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.569561 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.569763 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.572457 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:33.572547 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:33.572635 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:33.572660 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:33.572688 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.572793 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0908 14:18:33.578262 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.581906 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.582019 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.591724 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:33.595776 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.595896 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.595985 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.635577 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.823486 29538 iamrole.go:89] ListRoles: { } I0908 14:18:33.825874 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:33.826133 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } W0908 14:18:33.826363 29538 api.go:53] PageSize not implemented I0908 14:18:33.826338 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:33.826442 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:33.826610 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:33.826766 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.826879 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:33.826948 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:33.827092 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.827222 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:33.827256 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:33.827307 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:33.827460 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.827537 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.828712 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:33.828827 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0908 14:18:33.828985 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.829100 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:33.829131 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.829254 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:33.829313 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:33.829197 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.829330 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:33.829392 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.829397 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:33.829480 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.829493 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:33.829549 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:33.829749 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:33.829813 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:33.829960 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:33.830037 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.830180 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:33.830324 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:33.830384 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.830570 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.830731 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:33.830878 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:33.830916 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.830998 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:33.834433 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:33.834714 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.834764 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:33.834863 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:33.834828 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.834881 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.834998 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.835065 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.835123 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:33.835157 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:33.835202 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:33.835140 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.835386 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.835659 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.835882 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.835994 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.836088 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.837067 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.837295 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:33.837432 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:33.842585 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:33.842691 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:33.842793 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:33.842817 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:33.842840 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:33.843026 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0908 14:18:33.843086 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:33.843121 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:33.843180 29538 dns.go:238] Pre-creating DNS records I0908 14:18:33.851324 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:33.856190 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.856298 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.856360 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:33.896371 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.078170 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.080969 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:34.081182 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0908 14:18:34.081391 29538 api.go:53] PageSize not implemented I0908 14:18:34.081371 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.081539 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.081645 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:34.081678 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:34.081705 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:34.081783 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:34.081831 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.081883 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:34.081984 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:34.082075 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.082234 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.082371 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:34.084293 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:34.084467 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:34.084431 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0908 14:18:34.084562 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.084584 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:34.084626 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.084724 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:34.084824 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:34.084819 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.084761 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.084975 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.085087 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.085255 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:34.085302 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.085434 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:34.085603 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:34.085673 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.085851 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:34.086107 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.086172 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:34.086244 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.086425 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.086585 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:34.086761 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.086874 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:34.086992 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.090796 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:34.090982 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.091122 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:34.091214 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:34.091233 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.091279 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.091313 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.091353 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.091390 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.091119 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.091422 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.091532 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.091689 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.092187 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.092382 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.092452 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.092591 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.092684 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.092764 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.092905 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.095087 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:34.095179 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:34.095271 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:34.095296 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:34.095321 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.095392 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0908 14:18:34.098547 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.102122 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.102217 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.102280 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.111884 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:34.115860 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.115956 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.116015 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.116068 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.158291 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.350371 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.352820 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:34.353046 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.353189 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:34.353225 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:34.353290 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.353338 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0908 14:18:34.353397 29538 api.go:53] PageSize not implemented I0908 14:18:34.353499 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.353657 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:34.353696 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.354004 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.354183 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:34.354343 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:34.354486 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:34.354626 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:34.356184 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:34.356354 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.356456 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:34.356554 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.356594 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:34.356524 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.356646 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:34.356713 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.356674 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:34.356776 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:34.356821 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.356972 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.357166 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:34.357234 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.358409 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:34.358666 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:34.358723 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.358861 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.358987 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.359112 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:34.359284 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.359501 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:34.359791 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:34.359876 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.360010 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.360343 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:34.365653 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:34.365929 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.365985 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:34.366082 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:34.366103 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.366160 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.366201 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.366047 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.366243 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.366282 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.366314 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.366503 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.366697 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.366950 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.367079 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.367191 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.367318 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.367452 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.367601 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.367665 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.370863 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:34.370961 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:34.371048 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:34.371073 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:34.371096 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.371181 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0908 14:18:34.371261 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.371318 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:34.371422 29538 dns.go:238] Pre-creating DNS records I0908 14:18:34.379760 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0908 14:18:34.383843 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.383958 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.384023 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.384078 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.423451 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.616165 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.618543 29538 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0908 14:18:34.618789 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.618966 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0908 14:18:34.619070 29538 api.go:53] PageSize not implemented I0908 14:18:34.619040 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.619244 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.619450 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0908 14:18:34.619590 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:34.619625 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:34.619662 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0908 14:18:34.619820 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0908 14:18:34.619920 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0908 14:18:34.620013 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0908 14:18:34.620107 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0908 14:18:34.620196 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.621251 29538 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0908 14:18:34.621404 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:34.621385 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.621516 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.621649 29538 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:34.621610 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0908 14:18:34.621692 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.621695 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.621768 29538 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0908 14:18:34.621853 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.621856 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.622075 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:34.622146 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.622267 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.622406 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.622536 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:34.622573 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0908 14:18:34.622604 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.622715 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.622832 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.622914 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0908 14:18:34.623070 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.623117 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0908 14:18:34.623341 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.623388 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0908 14:18:34.623619 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:34.627238 29538 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0908 14:18:34.627465 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.627493 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:34.627586 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:34.627617 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.627594 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.627679 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.627750 29538 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.627807 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0908 14:18:34.627796 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.627861 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0908 14:18:34.627896 29538 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0908 14:18:34.628112 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.628320 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.628605 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.628736 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.628967 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:34.629094 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.629256 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.629406 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.632041 29538 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0908 14:18:34.632139 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:34.632228 29538 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0908 14:18:34.632252 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0908 14:18:34.632279 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.632355 29538 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0908 14:18:34.632695 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:34.632750 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:34.632785 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:34.632817 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:34.632857 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:34.632892 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:34.632925 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:34.632961 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:34.633004 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:34.633045 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:34.633088 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:34.633125 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:34.633159 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:34.633200 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:34.633233 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:34.633271 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:34.633309 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:34.633346 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:34.633383 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:34.633421 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:34.633455 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:34.633493 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0908 14:18:34.633531 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:34.633564 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:34.633599 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:34.633637 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:34.633675 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:34.633711 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:34.636821 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:34.636855 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:34.636888 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.636951 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0908 14:18:34.637018 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637079 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637129 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637190 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637233 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637302 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0908 14:18:34.637366 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637440 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0908 14:18:34.637481 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0908 14:18:34.637538 29538 api.go:53] PageSize not implemented I0908 14:18:34.637559 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:34.637585 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:34.637599 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0908 14:18:34.637638 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:34.637664 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:34.637683 29538 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" ] } I0908 14:18:34.637755 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.637782 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:34.637822 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:34.637859 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.637902 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:34.637951 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:34.637999 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0908 14:18:34.638524 29538 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0908 14:18:34.638561 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:nodes.complex.example.com ok I0908 14:18:34.638620 29538 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0908 14:18:34.638659 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0908 14:18:34.638694 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0908 14:18:34.638710 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0908 14:18:34.638782 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } security-group:sg-3 ok I0908 14:18:34.638718 29538 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0908 14:18:34.638827 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok I0908 14:18:34.638819 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0908 14:18:34.638705 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0908 14:18:34.638883 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } subnet:subnet-2 ok iam-instance-profile:nodes.complex.example.com ok internet-gateway:igw-2 ok I0908 14:18:34.638909 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.638890 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0908 14:18:34.638966 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0908 14:18:34.638957 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0908 14:18:34.639314 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:34.639336 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:34.639366 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.639419 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W0908 14:18:34.639618 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:34.639638 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:34.639668 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:34.639738 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:34.639828 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:34.639879 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0908 14:18:34.639863 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0908 14:18:34.639966 29538 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0908 14:18:34.639982 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0908 14:18:34.640014 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0908 14:18:34.640059 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0908 14:18:34.640152 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0908 14:18:34.640169 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0908 14:18:34.640203 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0908 14:18:34.640287 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:34.640370 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:34.640449 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:34.640776 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.73s) === RUN TestLifecycleExternalLB I0908 14:18:34.641079 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:34.641693 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:34.641741 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:34.641787 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:34.641833 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:34.641873 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:34.641901 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:34.641935 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:34.641961 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:34.642017 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0908 14:18:34.672492 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:34.675851 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.675960 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.718365 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.815765 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.817228 29538 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0908 14:18:34.817414 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0908 14:18:34.817556 29538 api.go:53] PageSize not implemented I0908 14:18:34.817541 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0908 14:18:34.817685 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0908 14:18:34.817682 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0908 14:18:34.817599 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0908 14:18:34.817831 29538 api.go:53] PageSize not implemented I0908 14:18:34.817808 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } W0908 14:18:34.817935 29538 api.go:53] PageSize not implemented I0908 14:18:34.817762 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0908 14:18:34.817937 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:34.820481 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:34.820150 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:34.820598 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.820912 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:34.818008 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0908 14:18:34.821028 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:34.821124 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0908 14:18:34.821224 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.821302 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:34.817993 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.821681 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:34.821692 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.821754 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.821805 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.821956 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:34.821987 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:34.822193 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:34.822280 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.822805 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.823270 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:34.823448 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:34.823569 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:34.823736 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:34.826025 29538 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0908 14:18:34.826157 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.826316 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:34.826401 29538 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0908 14:18:34.826373 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.826467 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0908 14:18:34.826582 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.826596 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:34.826551 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0908 14:18:34.826747 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:34.826792 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:34.826785 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0908 14:18:34.826847 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0908 14:18:34.827158 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:34.827215 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0908 14:18:34.827461 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] }], VpcId: "vpc-1" } I0908 14:18:34.827597 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.827703 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.827796 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:34.827777 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0908 14:18:34.827890 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:34.827940 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.827989 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:34.828092 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:34.828139 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:34.828275 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0908 14:18:34.828294 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:34.828393 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.828439 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:34.828489 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I0908 14:18:34.829514 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0908 14:18:34.831802 29538 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0908 14:18:34.831988 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.832163 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:34.832250 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:34.832314 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.832590 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I0908 14:18:34.832830 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.832979 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.833130 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.833358 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:34.833505 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.833648 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0908 14:18:34.833804 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.833911 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.834020 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.834128 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:34.834295 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.834441 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.834574 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:34.834717 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.834861 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0908 14:18:34.835083 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0908 14:18:34.835243 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:34.835305 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.835416 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.835487 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:34.837126 29538 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0908 14:18:34.837388 29538 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0908 14:18:34.837436 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.837484 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:34.837541 29538 dns.go:238] Pre-creating DNS records I0908 14:18:34.837570 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:34.843572 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:34.846450 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.846548 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:34.885993 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:34.989009 29538 iamrole.go:89] ListRoles: { } I0908 14:18:34.990675 29538 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0908 14:18:34.990860 29538 api.go:53] PageSize not implemented W0908 14:18:34.991052 29538 api.go:53] PageSize not implemented I0908 14:18:34.990956 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.991149 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:34.991122 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:34.991219 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:34.991258 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:34.991300 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:34.991319 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.991404 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0908 14:18:34.991528 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0908 14:18:34.991252 29538 api.go:53] PageSize not implemented I0908 14:18:34.991583 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.991524 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0908 14:18:34.991715 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0908 14:18:34.991765 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.991857 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:34.991886 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.991926 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:34.991996 29538 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.992129 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0908 14:18:34.994124 29538 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0908 14:18:34.994281 29538 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0908 14:18:34.994420 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0908 14:18:34.994380 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0908 14:18:34.994539 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0908 14:18:34.994749 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:34.994842 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:34.994953 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.995111 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.995218 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.995348 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:34.995486 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:34.995526 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:34.999342 29538 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0908 14:18:34.999524 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.999678 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:34.999839 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.000000 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.000257 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:35.000566 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.000770 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.002912 29538 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0908 14:18:35.003181 29538 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0908 14:18:35.006460 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:35.006527 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:35.006566 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:35.006599 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:35.006637 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:35.006671 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:35.006704 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:35.006738 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:35.006776 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:35.006811 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:35.006849 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:35.006894 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:35.006927 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:35.006960 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:35.006996 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:35.007029 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.007063 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.007099 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:35.007134 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:35.007165 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:35.007195 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:35.007226 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.007265 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.007324 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:35.007359 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:35.008736 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:35.008764 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:35.008800 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.008851 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0908 14:18:35.008907 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.008963 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009008 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009063 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009103 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009142 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0908 14:18:35.009186 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009236 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0908 14:18:35.009275 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0908 14:18:35.009328 29538 api.go:53] PageSize not implemented I0908 14:18:35.009350 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:35.009377 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:35.009396 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:35.009419 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:35.009435 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:35.009483 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.009512 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.009562 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:35.009598 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.009641 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:35.009683 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:35.009715 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:35.010128 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-config:lt-1 ok autoscaling-group:nodes.externallb.example.com ok I0908 14:18:35.010210 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0908 14:18:35.010289 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0908 14:18:35.010323 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-config:lt-2 ok internet-gateway:igw-2 ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0908 14:18:35.010392 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:35.010344 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0908 14:18:35.010470 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0908 14:18:35.010478 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.010538 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0908 14:18:35.010466 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0908 14:18:35.010565 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0908 14:18:35.010807 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.010837 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:35.010874 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.010933 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0908 14:18:35.011044 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.011058 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:35.011098 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:35.011195 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0908 14:18:35.011269 29538 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0908 14:18:35.011238 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:35.011287 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0908 14:18:35.011344 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0908 14:18:35.011778 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0908 14:18:35.011861 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0908 14:18:35.011880 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0908 14:18:35.011912 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0908 14:18:35.011987 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:35.012067 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:35.012138 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:35.012417 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I0908 14:18:35.012705 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:35.012948 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:35.013000 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:35.013044 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:35.013095 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:35.013137 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:35.013168 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:35.013208 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:35.013235 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:35.013290 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0908 14:18:35.043041 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.046348 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.046445 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.086607 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.184336 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.185681 29538 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0908 14:18:35.186111 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.186368 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.186521 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0908 14:18:35.186442 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.186634 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0908 14:18:35.189664 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0908 14:18:35.189640 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.189289 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:35.189815 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0908 14:18:35.189981 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.190331 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.190457 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.190838 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:35.191086 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:35.191148 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.192065 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.192134 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:35.193344 29538 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0908 14:18:35.193500 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0908 14:18:35.193600 29538 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0908 14:18:35.193541 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0908 14:18:35.193677 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0908 14:18:35.193774 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:35.193932 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0908 14:18:35.194012 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0908 14:18:35.194257 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0908 14:18:35.194472 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.194484 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0908 14:18:35.194608 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.194686 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.194808 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.194863 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.194921 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:35.194981 29538 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0908 14:18:35.195059 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.195888 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0908 14:18:35.199422 29538 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0908 14:18:35.199613 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0908 14:18:35.199880 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.200035 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.200188 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.200302 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.200455 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.200775 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.201070 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.201249 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.201392 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0908 14:18:35.201680 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.201853 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0908 14:18:35.202131 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.202411 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.202666 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.202915 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.203163 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.203408 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0908 14:18:35.204929 29538 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0908 14:18:35.205260 29538 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0908 14:18:35.205313 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.205358 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.205414 29538 dns.go:238] Pre-creating DNS records I0908 14:18:35.205444 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.211101 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.213894 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.214003 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.255057 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.357375 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.359007 29538 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0908 14:18:35.359200 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:35.359315 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.359520 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.359753 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.359859 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.360040 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:35.360121 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:35.360150 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:35.362269 29538 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0908 14:18:35.362535 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0908 14:18:35.362536 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0908 14:18:35.362653 29538 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0908 14:18:35.362736 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.362800 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0908 14:18:35.362960 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0908 14:18:35.363147 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.363216 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:35.371977 29538 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0908 14:18:35.372219 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.372497 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.372860 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.373039 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:35.374655 29538 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0908 14:18:35.375048 29538 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0908 14:18:35.378905 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:35.379029 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:35.379085 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:35.379123 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:35.379154 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:35.379187 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:35.379217 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:35.379257 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:35.379296 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:35.379337 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:35.379368 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:35.379398 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:35.379428 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.379461 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.379491 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:35.379525 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:35.379556 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:35.379587 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.379622 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.379656 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:35.381106 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:35.381133 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:35.381170 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381219 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0908 14:18:35.381278 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381401 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381467 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381530 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381573 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381606 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0908 14:18:35.381642 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381681 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0908 14:18:35.381715 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0908 14:18:35.381763 29538 api.go:53] PageSize not implemented I0908 14:18:35.381787 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:35.381813 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:35.381832 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:35.381857 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:35.381876 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:35.381926 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.381955 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.382006 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:35.382040 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.382087 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:35.382126 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:35.382487 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:nodes.sharedsubnet.example.com ok volume:vol-1 ok autoscaling-config:lt-2 ok I0908 14:18:35.382568 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0908 14:18:35.382606 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0908 14:18:35.382660 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } keypair:key-1 ok iam-instance-profile:nodes.sharedsubnet.example.com ok I0908 14:18:35.382682 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:35.382742 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.382730 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0908 14:18:35.382830 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0908 14:18:35.382630 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedsubnet.example.com. ok I0908 14:18:35.382843 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0908 14:18:35.383047 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.383076 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:35.383125 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.383206 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0908 14:18:35.383448 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.383470 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0908 14:18:35.383544 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0908 14:18:35.383598 29538 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0908 14:18:35.383614 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0908 14:18:35.383649 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0908 14:18:35.383703 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0908 14:18:35.383798 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0908 14:18:35.383825 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0908 14:18:35.383877 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0908 14:18:35.384171 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I0908 14:18:35.384441 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:35.384697 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:35.384748 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:35.384794 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:35.384844 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:35.384890 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:35.384922 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:35.384959 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:35.384986 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:35.385040 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0908 14:18:35.413880 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.416929 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.417022 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.417085 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.461887 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.568336 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.570491 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:18:35.570795 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.570924 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0908 14:18:35.571001 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.571201 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.571230 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:35.571335 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.571413 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.571598 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.571643 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0908 14:18:35.571689 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.571813 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.571865 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.571888 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0908 14:18:35.571921 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:35.572005 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:35.572036 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:35.572085 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.572143 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.572296 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.573358 29538 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0908 14:18:35.573476 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0908 14:18:35.573615 29538 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0908 14:18:35.573735 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0908 14:18:35.573690 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0908 14:18:35.573839 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.573888 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0908 14:18:35.574011 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.574178 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.574290 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.574155 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.574348 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.574395 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0908 14:18:35.574374 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.574602 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:35.574631 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.574659 29538 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/role/internal-elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" }] } I0908 14:18:35.574729 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0908 14:18:35.574730 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.574822 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.574867 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.574905 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0908 14:18:35.575071 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:35.575234 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0908 14:18:35.575435 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0908 14:18:35.575693 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.575824 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:35.575880 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:35.575920 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:35.575975 29538 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" }] } I0908 14:18:35.576056 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.576196 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:35.576252 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:35.576295 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.576376 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:35.576418 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:35.577920 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.581037 29538 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0908 14:18:35.581230 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0908 14:18:35.581372 29538 api.go:53] PageSize not implemented W0908 14:18:35.581469 29538 api.go:53] PageSize not implemented I0908 14:18:35.581425 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:35.581548 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.581607 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0908 14:18:35.581694 29538 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0908 14:18:35.581761 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0908 14:18:35.581796 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0908 14:18:35.581756 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.581862 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0908 14:18:35.581922 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.581946 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0908 14:18:35.582033 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.582039 29538 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0908 14:18:35.582107 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0908 14:18:35.582142 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0908 14:18:35.582196 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0908 14:18:35.582113 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.582442 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.582604 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.582848 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.583052 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.583237 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.583344 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.583535 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.583639 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.583742 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.583870 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.584075 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.584240 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0908 14:18:35.584359 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.584512 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.584642 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.584771 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.584903 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.585044 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:35.585161 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0908 14:18:35.585286 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0908 14:18:35.585415 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0908 14:18:35.585513 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0908 14:18:35.585652 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" }] }] } I0908 14:18:35.585914 29538 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0908 14:18:35.586129 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:35.586305 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:35.586427 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:18:35.586465 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.586494 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.586538 29538 dns.go:238] Pre-creating DNS records I0908 14:18:35.586563 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.594766 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.600357 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.600467 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.600533 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.641401 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.738697 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.740872 29538 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0908 14:18:35.741246 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.741438 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.741554 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:35.741532 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.741738 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:35.741871 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:35.741929 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:35.741998 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.747422 29538 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0908 14:18:35.747564 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0908 14:18:35.747707 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:35.748587 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0908 14:18:35.748652 29538 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0908 14:18:35.748621 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0908 14:18:35.748902 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.748982 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0908 14:18:35.749098 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0908 14:18:35.749258 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:35.749321 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0908 14:18:35.749531 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:35.749591 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0908 14:18:35.749712 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0908 14:18:35.749888 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.749932 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0908 14:18:35.750096 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:35.754454 29538 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0908 14:18:35.754630 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0908 14:18:35.754837 29538 api.go:53] PageSize not implemented I0908 14:18:35.754788 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.754873 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0908 14:18:35.754979 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0908 14:18:35.754984 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.755017 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0908 14:18:35.755189 29538 api.go:53] PageSize not implemented I0908 14:18:35.755159 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.755223 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0908 14:18:35.755350 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0908 14:18:35.755397 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0908 14:18:35.755390 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.755627 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.757953 29538 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0908 14:18:35.758080 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:35.758194 29538 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" W0908 14:18:35.758162 29538 api.go:53] PageSize not implemented I0908 14:18:35.758241 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } W0908 14:18:35.758455 29538 api.go:53] PageSize not implemented I0908 14:18:35.758497 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0908 14:18:35.758634 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0908 14:18:35.758747 29538 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0908 14:18:35.762618 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:35.762677 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:35.762715 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:35.762749 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:35.762784 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:35.762816 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:35.762848 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:35.762884 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:35.762922 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:35.762957 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0908 14:18:35.763002 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:35.763032 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:35.763063 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:35.763093 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.763129 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.763166 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:35.763200 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:35.763236 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:35.763270 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:35.763304 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:35.763338 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:35.763370 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.763405 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.763439 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:35.765075 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:35.765104 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:35.765144 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765207 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0908 14:18:35.765282 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765340 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765382 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765425 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765469 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765504 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0908 14:18:35.765541 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765581 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0908 14:18:35.765614 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0908 14:18:35.765658 29538 api.go:53] PageSize not implemented I0908 14:18:35.765675 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0908 14:18:35.765731 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:35.765759 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:35.765777 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:35.765802 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:35.765822 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:35.765870 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.765900 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.765948 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:35.765987 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.766030 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:35.766072 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:35.766560 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:35.766636 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0908 14:18:35.766620 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } autoscaling-config:lt-2 ok load-balancer:api-private-shared-subnet-n2f8ak ok I0908 14:18:35.766656 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.766720 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0908 14:18:35.766764 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } autoscaling-group:nodes.private-shared-subnet.example.com ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0908 14:18:35.766806 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.766621 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0908 14:18:35.766868 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0908 14:18:35.766881 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok iam-instance-profile:masters.private-shared-subnet.example.com ok I0908 14:18:35.766779 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0908 14:18:35.766992 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.766955 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.767036 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0908 14:18:35.767009 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0908 14:18:35.767100 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.767204 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0908 14:18:35.767334 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.767360 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-3 ok I0908 14:18:35.767416 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0908 14:18:35.767486 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:35.767537 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0908 14:18:35.767649 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.767663 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0908 14:18:35.767741 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.767798 29538 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0908 14:18:35.767814 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0908 14:18:35.767794 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:35.767857 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.767942 29538 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0908 14:18:35.767880 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0908 14:18:35.767998 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.768015 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0908 14:18:35.767958 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0908 14:18:35.768052 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:35.768072 29538 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0908 14:18:35.768129 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0908 14:18:35.768100 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0908 14:18:35.768177 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:35.768192 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0908 14:18:35.768173 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.768232 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0908 14:18:35.768249 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0908 14:18:35.768281 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0908 14:18:35.768557 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0908 14:18:35.768894 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:35.769173 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:35.769222 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:35.769268 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:35.769315 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:35.769362 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:35.769395 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:35.769430 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:35.769457 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:35.769510 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0908 14:18:35.797826 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.800852 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.800946 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.801009 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.840945 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.945729 29538 iamrole.go:89] ListRoles: { } I0908 14:18:35.948168 29538 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0908 14:18:35.948521 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.948626 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:35.948779 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0908 14:18:35.948948 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:35.949039 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:35.949069 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:35.948959 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0908 14:18:35.949112 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:35.949157 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:35.949137 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0908 14:18:35.949323 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.949473 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:35.949522 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:35.949543 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0908 14:18:35.949672 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0908 14:18:35.949659 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:35.949775 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0908 14:18:35.949817 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:35.949874 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0908 14:18:35.952342 29538 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0908 14:18:35.952570 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0908 14:18:35.952973 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.953148 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:35.953219 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0908 14:18:35.953254 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:35.953277 29538 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0908 14:18:35.953384 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0908 14:18:35.953342 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:35.953514 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0908 14:18:35.953584 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0908 14:18:35.953861 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0908 14:18:35.954207 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:35.955001 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0908 14:18:35.953911 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0908 14:18:35.955142 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:35.955194 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:35.954156 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0908 14:18:35.955240 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0908 14:18:35.955164 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0908 14:18:35.955404 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0908 14:18:35.955448 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.955543 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0908 14:18:35.955567 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0908 14:18:35.956300 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:35.956412 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0908 14:18:35.956530 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0908 14:18:35.956642 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:35.956593 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0908 14:18:35.956740 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:35.956780 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.956879 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:35.956931 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:35.956981 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.957069 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:35.957112 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:35.957150 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0908 14:18:35.957262 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:35.957345 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:35.957390 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:35.957551 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:35.957534 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0908 14:18:35.957639 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:35.957685 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:35.957743 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0908 14:18:35.957835 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:35.957878 29538 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0908 14:18:35.957956 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:35.957821 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0908 14:18:35.961152 29538 executor.go:111] Tasks: 71 done / 113 total; 36 can run W0908 14:18:35.961372 29538 api.go:53] PageSize not implemented W0908 14:18:35.961442 29538 api.go:53] PageSize not implemented I0908 14:18:35.961414 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.961574 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0908 14:18:35.961694 29538 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0908 14:18:35.961786 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0908 14:18:35.961728 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:35.961847 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0908 14:18:35.961925 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0908 14:18:35.962047 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0908 14:18:35.962060 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0908 14:18:35.962179 29538 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0908 14:18:35.962264 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0908 14:18:35.962295 29538 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0908 14:18:35.962342 29538 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0908 14:18:35.962372 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" }] }] } I0908 14:18:35.962591 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:35.962741 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0908 14:18:35.962814 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:35.962935 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0908 14:18:35.963080 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.963327 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0908 14:18:35.963503 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:35.963574 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0908 14:18:35.963689 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.963822 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:35.963983 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0908 14:18:35.964128 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0908 14:18:35.964336 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.964496 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.964593 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.964724 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0908 14:18:35.964840 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" }] }] } I0908 14:18:35.964975 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0908 14:18:35.965021 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" }] }] } I0908 14:18:35.965160 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:35.965199 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.965333 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:35.965371 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:35.965473 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.965599 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.965745 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.965824 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0908 14:18:35.965920 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.966058 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.966190 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0908 14:18:35.966236 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.966375 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0908 14:18:35.966488 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0908 14:18:35.966621 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.966666 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.966742 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:35.966789 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:35.966857 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.966992 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.967136 29538 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0908 14:18:35.969014 29538 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0908 14:18:35.969197 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:35.969270 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:35.969297 29538 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:35.969429 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:35.969545 29538 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0908 14:18:35.969666 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:35.969714 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:35.969757 29538 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0908 14:18:35.969830 29538 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0908 14:18:35.969891 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:35.970022 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0908 14:18:35.970091 29538 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0908 14:18:35.970134 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:35.970164 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.970211 29538 dns.go:238] Pre-creating DNS records I0908 14:18:35.970235 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:35.976562 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0908 14:18:35.979546 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.979639 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:35.979703 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.019548 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.119802 29538 iamrole.go:89] ListRoles: { } I0908 14:18:36.122075 29538 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0908 14:18:36.122442 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.122730 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.122777 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:36.122823 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0908 14:18:36.122968 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0908 14:18:36.123023 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0908 14:18:36.123091 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.123288 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.124590 29538 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0908 14:18:36.124767 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0908 14:18:36.124982 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:36.125135 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0908 14:18:36.125072 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0908 14:18:36.125175 29538 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0908 14:18:36.125371 29538 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0908 14:18:36.125204 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0908 14:18:36.125475 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:36.125545 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:36.125694 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0908 14:18:36.125849 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0908 14:18:36.126045 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0908 14:18:36.126110 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.126265 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0908 14:18:36.126320 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.126512 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.126650 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0908 14:18:36.126797 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:36.126869 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.127022 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:36.129613 29538 executor.go:111] Tasks: 71 done / 113 total; 36 can run W0908 14:18:36.129796 29538 api.go:53] PageSize not implemented I0908 14:18:36.129758 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.129828 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0908 14:18:36.129941 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0908 14:18:36.129874 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.130062 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.129985 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } W0908 14:18:36.130282 29538 api.go:53] PageSize not implemented I0908 14:18:36.130307 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0908 14:18:36.130398 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0908 14:18:36.130366 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0908 14:18:36.130434 29538 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0908 14:18:36.130493 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:36.130709 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.130828 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.130943 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:36.131091 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.131387 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:36.131555 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0908 14:18:36.131843 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0908 14:18:36.133293 29538 executor.go:111] Tasks: 107 done / 113 total; 5 can run W0908 14:18:36.133402 29538 api.go:53] PageSize not implemented I0908 14:18:36.133397 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.133404 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0908 14:18:36.133479 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:36.133525 29538 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0908 14:18:36.133428 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0908 14:18:36.133704 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0908 14:18:36.133766 29538 api.go:53] PageSize not implemented I0908 14:18:36.133788 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0908 14:18:36.133998 29538 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0908 14:18:36.134066 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.134174 29538 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0908 14:18:36.138047 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:36.138108 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:36.138146 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:36.138181 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:36.138218 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:36.138254 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:36.138289 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:36.138325 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:36.138364 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:36.138401 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0908 14:18:36.138443 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0908 14:18:36.138481 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:36.138514 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:36.138546 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:36.138584 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:36.138616 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0908 14:18:36.138651 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:36.138686 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:36.138723 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0908 14:18:36.138760 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0908 14:18:36.138794 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0908 14:18:36.138829 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:36.138866 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:36.138898 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0908 14:18:36.138934 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:36.138968 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:36.139000 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:36.139035 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:36.139071 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:36.140640 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:36.140667 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:36.140705 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.140768 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0908 14:18:36.140846 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.140903 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.140945 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.140990 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.141030 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.141073 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0908 14:18:36.141125 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.141175 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:36.141201 29538 natgateway.go:117] DescribeNatGateways: { } I0908 14:18:36.141243 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0908 14:18:36.141279 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0908 14:18:36.141325 29538 api.go:53] PageSize not implemented I0908 14:18:36.141342 29538 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0908 14:18:36.141393 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:36.141421 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:36.141439 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:36.141464 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:36.141484 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0908 14:18:36.141540 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.141565 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:36.141611 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:36.141647 29538 iamrole.go:89] ListRoles: { } I0908 14:18:36.141696 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:36.141748 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:36.141783 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0908 14:18:36.141819 29538 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0908 14:18:36.141856 29538 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0908 14:18:36.142439 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:36.142512 29538 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0908 14:18:36.142499 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } nat-gateway:nat-1 ok autoscaling-config:lt-1 ok I0908 14:18:36.142571 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.private-shared-ip.example.com ok load-balancer:api-private-shared-ip-exa-ohatqj ok I0908 14:18:36.142635 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } keypair:key-1 ok autoscaling-group:bastion.private-shared-ip.example.com ok I0908 14:18:36.142618 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } autoscaling-config:lt-3 ok I0908 14:18:36.142697 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0908 14:18:36.142793 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.142749 29538 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I0908 14:18:36.142595 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok I0908 14:18:36.142747 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0908 14:18:36.142812 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0908 14:18:36.142894 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0908 14:18:36.142916 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:36.142955 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0908 14:18:36.142988 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0908 14:18:36.143034 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0908 14:18:36.142980 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0908 14:18:36.143180 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.143202 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0908 14:18:36.143251 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0908 14:18:36.143301 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0908 14:18:36.143406 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.143429 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0908 14:18:36.143461 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:36.143563 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0908 14:18:36.143617 29538 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0908 14:18:36.143584 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:36.143632 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0908 14:18:36.143680 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0908 14:18:36.143713 29538 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0908 14:18:36.143766 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0908 14:18:36.143742 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0908 14:18:36.143829 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0908 14:18:36.143847 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0908 14:18:36.143887 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok W0908 14:18:36.143833 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.143922 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0908 14:18:36.143993 29538 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0908 14:18:36.144006 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0908 14:18:36.144036 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0908 14:18:36.143960 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0908 14:18:36.144084 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:36.144146 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0908 14:18:36.144253 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.144272 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:36.144318 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0908 14:18:36.144405 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:36.144464 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0908 14:18:36.144725 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.38s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0908 14:18:36.144980 29538 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:18:36.145216 29538 internetgateways.go:66] CreateInternetGateway: { } I0908 14:18:36.145264 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:18:36.145307 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:18:36.145355 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:18:36.145394 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:18:36.145429 29538 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:18:36.145465 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:18:36.145492 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:18:36.145548 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0908 14:18:36.176446 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:36.179471 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.179573 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.222477 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.352198 29538 iamrole.go:89] ListRoles: { } I0908 14:18:36.353887 29538 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0908 14:18:36.354288 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:36.354378 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.354467 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.354515 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.354622 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] } I0908 14:18:36.354652 29538 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.354821 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.354921 29538 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.355054 29538 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" }] } I0908 14:18:36.355095 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.355317 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.355602 29538 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.355595 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.355925 29538 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.356119 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:36.356199 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:36.356426 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:36.356521 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0908 14:18:36.356577 29538 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0908 14:18:36.356614 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:36.356670 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.356707 29538 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0908 14:18:36.356839 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:36.357652 29538 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0908 14:18:36.357861 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.358039 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0908 14:18:36.358061 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.358091 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.358112 29538 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:36.358300 29538 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0908 14:18:36.358216 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.358361 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0908 14:18:36.358394 29538 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.358503 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.358547 29538 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0908 14:18:36.358659 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:36.358603 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.358709 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.358890 29538 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:18:36.358960 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0908 14:18:36.359237 29538 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.359354 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:36.359412 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0908 14:18:36.359617 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:36.359715 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:36.359772 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:36.359814 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:36.359868 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:36.359910 29538 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0908 14:18:36.359947 29538 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0908 14:18:36.360050 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:36.360096 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:36.360138 29538 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" } ] } I0908 14:18:36.360789 29538 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.362951 29538 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0908 14:18:36.363148 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.363340 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.363565 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0908 14:18:36.363717 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.364030 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.364231 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.364489 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.364602 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.364681 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.364815 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.364955 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.365185 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.365280 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.365475 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.365588 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.365730 29538 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0908 14:18:36.365945 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.366167 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0908 14:18:36.366219 29538 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0908 14:18:36.366261 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.366407 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.366550 29538 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:36.366604 29538 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0908 14:18:36.366653 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.366809 29538 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0908 14:18:36.367819 29538 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0908 14:18:36.368093 29538 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0908 14:18:36.368301 29538 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0908 14:18:36.368352 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.368392 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:36.368448 29538 dns.go:238] Pre-creating DNS records I0908 14:18:36.368479 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:36.378071 29538 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0908 14:18:36.381260 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.381366 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.421704 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.549830 29538 iamrole.go:89] ListRoles: { } I0908 14:18:36.552125 29538 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0908 14:18:36.552370 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.552556 29538 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0908 14:18:36.552741 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.552905 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.553004 29538 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0908 14:18:36.553109 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.553322 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:18:36.553372 29538 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:18:36.553424 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.553469 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.555365 29538 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0908 14:18:36.555501 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.555820 29538 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0908 14:18:36.555732 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0908 14:18:36.555910 29538 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0908 14:18:36.555980 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:36.556083 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.556314 29538 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:18:36.556387 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0908 14:18:36.556649 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:36.556764 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.556981 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.557136 29538 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:18:36.560728 29538 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0908 14:18:36.560897 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.561097 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.561237 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.561336 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.561482 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.561908 29538 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0908 14:18:36.562082 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.564040 29538 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0908 14:18:36.564271 29538 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0908 14:18:36.564388 29538 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0908 14:18:36.567192 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0908 14:18:36.567253 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0908 14:18:36.567317 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0908 14:18:36.567352 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0908 14:18:36.567392 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0908 14:18:36.567425 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0908 14:18:36.567456 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0908 14:18:36.567487 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0908 14:18:36.567521 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0908 14:18:36.567556 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0908 14:18:36.567593 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0908 14:18:36.567633 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0908 14:18:36.567667 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0908 14:18:36.567698 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0908 14:18:36.567733 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0908 14:18:36.567766 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:18:36.567799 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0908 14:18:36.567833 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:18:36.567870 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0908 14:18:36.567901 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0908 14:18:36.567932 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0908 14:18:36.567964 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0908 14:18:36.568001 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0908 14:18:36.568038 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:18:36.568073 29538 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0908 14:18:36.569581 29538 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0908 14:18:36.569609 29538 keypairs.go:95] DescribeKeyPairs: { } I0908 14:18:36.569651 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569705 29538 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569766 29538 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569826 29538 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569877 29538 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569934 29538 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.569975 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.570014 29538 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.570061 29538 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.570113 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0908 14:18:36.570155 29538 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0908 14:18:36.570210 29538 api.go:53] PageSize not implemented I0908 14:18:36.570231 29538 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0908 14:18:36.570258 29538 loadbalancers.go:34] PageSize not implemented I0908 14:18:36.570276 29538 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0908 14:18:36.570300 29538 targetgroups.go:35] PageSize not implemented I0908 14:18:36.570319 29538 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0908 14:18:36.570373 29538 zones.go:93] ListHostedZonesPages { } I0908 14:18:36.570401 29538 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0908 14:18:36.570451 29538 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0908 14:18:36.570487 29538 iamrole.go:89] ListRoles: { } I0908 14:18:36.570531 29538 internetgateways.go:101] DescribeInternetGateways: { } I0908 14:18:36.570574 29538 routetable.go:53] DescribeRouteTables: { } I0908 14:18:36.570629 29538 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0908 14:18:36.571058 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.571127 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0908 14:18:36.571107 29538 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0908 14:18:36.571200 29538 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.571255 29538 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0908 14:18:36.571199 29538 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0908 14:18:36.571308 29538 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-config:lt-2 ok keypair:key-1 ok internet-gateway:igw-2 ok I0908 14:18:36.571337 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0908 14:18:36.571389 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0908 14:18:36.571263 29538 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok I0908 14:18:36.571465 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0908 14:18:36.571604 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.571627 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0908 14:18:36.571664 29538 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0908 14:18:36.571717 29538 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0908 14:18:36.571910 29538 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0908 14:18:36.571931 29538 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0908 14:18:36.571984 29538 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0908 14:18:36.572073 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.572127 29538 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0908 14:18:36.572108 29538 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0908 14:18:36.572143 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0908 14:18:36.572175 29538 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0908 14:18:36.572209 29538 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.572250 29538 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0908 14:18:36.572263 29538 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0908 14:18:36.572291 29538 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0908 14:18:36.572351 29538 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0908 14:18:36.572428 29538 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0908 14:18:36.572496 29538 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0908 14:18:36.572761 29538 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.43s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.01s) PASS ok k8s.io/kops/cmd/kops 59.168s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.062s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0908 14:16:30.979688 5021 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.072s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.086s === RUN TestPodController I0908 14:17:07.179232 23704 pod.go:59] starting pod controller I0908 14:17:07.179534 23704 stoppable.go:62] shutting down controller I0908 14:17:07.179586 23704 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS I0908 14:17:07.179645 23704 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.024s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.065s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000115de0} route53_test.go:93: Got zone list: [0xc000115e10] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000115ee0} route53_test.go:93: Got zone list: [0xc000115f10] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc000115fb0} route53_test.go:170: Successfully added managed DNS zone: &{0xc00032b140 0xc00012a0a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000350110} route53_test.go:93: Got zone list: [0xc000350140] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a0c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000350240} route53_test.go:93: Got zone list: [0xc000350270] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a0e0} route53_test.go:187: Successfully added resource record set: {0xc000188100 0xc00012a100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003504f0} route53_test.go:93: Got zone list: [0xc000350520] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a188} route53_test.go:199: Successfully added resource record set: {0xc000188280 0xc00012a1a8} route53_test.go:118: Got 1 recordsets: [0xc0003506c0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003507f0} route53_test.go:93: Got zone list: [0xc000350820] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a240} route53_test.go:221: Successfully added resource record set: {0xc000188400 0xc00012a260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000188400 0xc00012a260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000350b80} route53_test.go:93: Got zone list: [0xc000350bb0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a318} route53_test.go:246: Successfully removed resource set {0xc000188600 0xc00012a338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000350e30} route53_test.go:93: Got zone list: [0xc000350e60] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a3c0} route53_test.go:264: Successfully removed resource set {0xc000188780 0xc00012a3e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000351110} route53_test.go:93: Got zone list: [0xc000351140] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a470} commontests.go:192: Successfully added resource record set: {0xc000188900 0xc00012a490} commontests.go:45: Correctly replaced resource record {0xc000188900 0xc00012a490} -> {0xc000188a00 0xc00012a4e0} commontests.go:142: Got 1 recordsets: [0xc000351450] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000351620} route53_test.go:93: Got zone list: [0xc000351650] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a590} commontests.go:192: Successfully added resource record set: {0xc000188c80 0xc00012a5b0} commontests.go:70: Correctly replaced resource record {0xc000188c80 0xc00012a5b0} -> {0xc000188d80 0xc00012a600} commontests.go:142: Got 1 recordsets: [0xc000351960] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000351b80} route53_test.go:93: Got zone list: [0xc000351bb0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a6c0} commontests.go:192: Successfully added resource record set: {0xc000189000 0xc00012a6e0} commontests.go:142: Got 2 recordsets: [0xc000351e20 0xc000351e40] commontests.go:142: Got 2 recordsets: [0xc000351eb0 0xc000351ed0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000362070} route53_test.go:93: Got zone list: [0xc0003620a0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032af60 0xc00012a7e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.012s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.005s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543340} clouddns_test.go:71: Got zone list: [0xc000572318] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc0005722e8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543440} clouddns_test.go:71: Got zone list: [0xc0005723c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572390} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543540} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000549040 0xc000572468} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543600} clouddns_test.go:71: Got zone list: [0xc000572510] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc0005724e0} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543720} clouddns_test.go:71: Got zone list: [0xc000572630] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572600} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543860} clouddns_test.go:71: Got zone list: [0xc000572780] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572750} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc0005439c0} clouddns_test.go:71: Got zone list: [0xc000572978] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572948} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543b20} clouddns_test.go:71: Got zone list: [0xc000572af8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572ac8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543c60} clouddns_test.go:71: Got zone list: [0xc000572c48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572c18} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543dc0} clouddns_test.go:71: Got zone list: [0xc000572df8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000572dc8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc000543f20} clouddns_test.go:71: Got zone list: [0xc0005730b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000573080} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc0000c81c0} clouddns_test.go:71: Got zone list: [0xc0005733c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000573398} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00012a6d8 0xc0000c83a0} clouddns_test.go:71: Got zone list: [0xc000573788] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000548f00 0xc000573758} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.113s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.059s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.081s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0908 14:17:38.188023 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.188335 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.188378 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.188413 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.188458 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.188499 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.188514 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.188541 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.188564 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.250385 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.11s) === RUN TestContainerdBuilder_Docker_19_03_14 I0908 14:17:38.294774 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.294990 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.295027 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.295064 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.295095 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.295121 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.295141 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.295165 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.295182 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.335565 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.05s) === RUN TestContainerdBuilder_Simple I0908 14:17:38.340991 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.341134 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.341161 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.341187 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.341212 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.341233 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.341248 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.341267 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.341280 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.342262 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.343609 29569 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0908 14:17:38.343626 29569 assetstore.go:111] bin/containerd I0908 14:17:38.343634 29569 assetstore.go:111] bin/containerd-shim I0908 14:17:38.343641 29569 assetstore.go:111] bin/containerd-shim-runc-v1 I0908 14:17:38.343648 29569 assetstore.go:111] bin/containerd-shim-runc-v2 I0908 14:17:38.343655 29569 assetstore.go:111] bin/containerd-stress I0908 14:17:38.343663 29569 assetstore.go:111] bin/ctr I0908 14:17:38.343710 29569 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0908 14:17:38.343724 29569 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0908 14:17:38.346690 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.346814 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.346838 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.346861 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.346887 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.346909 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.346942 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.346967 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.346979 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.363798 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.365070 29569 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.03s) === RUN TestContainerdBuilder_SkipInstall I0908 14:17:38.380405 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.380669 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.380699 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.380752 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.380799 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.380914 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.380945 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.380982 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.381007 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.382665 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.385114 29569 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0908 14:17:38.386453 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.386679 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.386720 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.386758 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.386803 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.386870 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.386898 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.386932 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.386955 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.509297 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.512277 29569 assetstore.go:108] Matching assets for "^docker/": I0908 14:17:38.512307 29569 assetstore.go:111] docker/containerd I0908 14:17:38.512322 29569 assetstore.go:111] docker/containerd-shim I0908 14:17:38.512334 29569 assetstore.go:111] docker/ctr I0908 14:17:38.512346 29569 assetstore.go:111] docker/docker I0908 14:17:38.512359 29569 assetstore.go:111] docker/docker-init I0908 14:17:38.512373 29569 assetstore.go:111] docker/docker-proxy I0908 14:17:38.512386 29569 assetstore.go:111] docker/dockerd I0908 14:17:38.512400 29569 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.13s) === RUN TestDockerBuilder_LogFlags I0908 14:17:38.518296 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.518529 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.518570 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.518623 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.518690 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.518765 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.518793 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.518825 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.518847 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.520825 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.523365 29569 assetstore.go:108] Matching assets for "^docker/": I0908 14:17:38.523391 29569 assetstore.go:111] docker/containerd I0908 14:17:38.523404 29569 assetstore.go:111] docker/containerd-shim I0908 14:17:38.523415 29569 assetstore.go:111] docker/ctr I0908 14:17:38.523426 29569 assetstore.go:111] docker/docker I0908 14:17:38.523437 29569 assetstore.go:111] docker/docker-init I0908 14:17:38.523448 29569 assetstore.go:111] docker/docker-proxy I0908 14:17:38.523462 29569 assetstore.go:111] docker/dockerd I0908 14:17:38.523472 29569 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.07s) === RUN TestDockerBuilder_SkipInstall I0908 14:17:38.586853 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.587103 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.587145 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.587185 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.587228 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.587326 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.587367 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.587402 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.587425 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.589098 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.591536 29569 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0908 14:17:38.591799 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.592010 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.592049 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.592086 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.592140 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.592211 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.592240 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.592273 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.592296 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.664647 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:38.666987 29569 assetstore.go:108] Matching assets for "^docker/": I0908 14:17:38.667016 29569 assetstore.go:111] docker/docker I0908 14:17:38.667031 29569 assetstore.go:111] docker/docker-containerd I0908 14:17:38.667043 29569 assetstore.go:111] docker/docker-containerd-ctr I0908 14:17:38.667055 29569 assetstore.go:111] docker/docker-containerd-shim I0908 14:17:38.667067 29569 assetstore.go:111] docker/docker-init I0908 14:17:38.667079 29569 assetstore.go:111] docker/docker-proxy I0908 14:17:38.667093 29569 assetstore.go:111] docker/docker-runc I0908 14:17:38.667105 29569 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.08s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0908 14:17:38.674625 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.674799 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.674835 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.674864 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.674909 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.674936 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.674952 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.674974 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.674990 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.677560 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0908 14:17:38.680215 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.680387 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.680424 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.680453 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.680484 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.680510 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.680542 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.680570 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.680587 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.682820 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.05s) === RUN TestKopsControllerBuilder I0908 14:17:38.729391 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.729584 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.729636 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.729675 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.729711 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.729740 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.729758 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.729784 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.729803 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.732049 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0908 14:17:38.738340 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.738481 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.738510 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.738535 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.738562 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.738583 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.738600 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.738621 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.738634 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.740129 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0908 14:17:38.749010 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.749154 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.749184 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.749209 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.749237 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.749264 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.749281 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.749303 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.749319 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.751182 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0908 14:17:38.759080 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.759214 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.759242 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.759267 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.759291 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.759315 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.759330 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.759350 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.759363 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.764416 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.04s) === RUN TestAwsIamAuthenticator I0908 14:17:38.803644 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.803882 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.803931 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.804013 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.804059 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.804096 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.804125 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.804159 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.804181 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.807227 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0908 14:17:38.817958 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:38.818089 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:38.818115 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:38.818139 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:38.818164 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:38.818184 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:38.818200 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:38.818219 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:38.818232 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:38.819731 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.38s) === RUN TestKubeAPIServerBuilderARM64 I0908 14:17:40.199139 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.199402 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.199455 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.199504 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.199551 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.199591 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.199620 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.199654 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.199679 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.202546 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.10s) === RUN TestKubeControllerManagerBuilder I0908 14:17:40.296133 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.296292 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.296328 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.296361 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.296392 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.296420 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.296438 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.296468 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.296485 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.298178 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.02s) === RUN TestKubeControllerManagerBuilderAMD64 I0908 14:17:40.318900 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.319036 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.319070 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.319098 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.319125 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.319150 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.319168 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.319191 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.319206 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.320894 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.15s) === RUN TestKubeControllerManagerBuilderARM64 I0908 14:17:40.472033 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.472329 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.472385 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.472444 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.472492 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.472530 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.472562 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.472598 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.472624 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.475584 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.17s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0908 14:17:40.638034 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.638259 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.638305 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.638348 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.638395 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.638430 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.638459 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.638491 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.638513 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.641207 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.10s) === RUN TestKubeProxyBuilderAMD64 I0908 14:17:40.739206 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.739431 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.739477 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.739520 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.739564 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.739600 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.739630 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.739664 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.739687 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.742483 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.08s) === RUN TestKubeProxyBuilderARM64 I0908 14:17:40.817337 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.817574 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.817617 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.817664 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.817707 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.817740 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.817769 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.817800 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.817822 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.820632 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.17s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0908 14:17:40.983628 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.983877 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.983925 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.984015 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.984044 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.984093 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.984123 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.984157 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.984185 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.987007 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0908 14:17:40.993626 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:40.993764 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:40.993791 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:40.993817 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:40.993842 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:40.993864 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:40.993881 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:40.993901 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:40.993916 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:40.995523 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.08s) === RUN TestKubeSchedulerBuilderARM64 I0908 14:17:41.071870 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.072031 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.072063 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.072089 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.072116 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.072140 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.072155 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.072180 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.072196 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.073826 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.12s) === RUN TestKubectlBuilder I0908 14:17:41.191701 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.191901 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.191941 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.191972 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.192006 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.192033 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.192049 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.192070 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.192095 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.194008 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:41.198018 29569 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0908 14:17:41.201147 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.201423 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.201510 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.201559 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.201614 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.201657 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.201685 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.201722 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.201751 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.207935 29569 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0908 14:17:41.213462 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.213626 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.213683 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.213722 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.213758 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.213793 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.213818 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.213850 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.213872 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.215714 29569 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0908 14:17:41.220403 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.220574 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.220623 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.220673 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.220705 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.220735 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.220757 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.220783 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.220801 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.222733 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:41.224732 29569 assetstore.go:108] Matching assets for "protokube$": I0908 14:17:41.224756 29569 assetstore.go:111] /opt/kops/bin/protokube I0908 14:17:41.224780 29569 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0908 14:17:41.224804 29569 assetstore.go:108] Matching assets for "channels$": I0908 14:17:41.224815 29569 assetstore.go:111] /opt/kops/bin/channels I0908 14:17:41.224825 29569 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0908 14:17:41.226441 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.226594 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.226627 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.226655 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.226684 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.226712 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.226733 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.226761 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.226777 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.228629 29569 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0908 14:17:41.231504 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.231708 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.231752 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.231801 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.231836 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.231883 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.231909 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.231941 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.231961 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.234240 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:41.236647 29569 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I0908 14:17:41.237253 29569 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:41.237450 29569 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:41.237493 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:41.237528 29569 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:41.237565 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:41.237598 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:41.237626 29569 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:41.237655 29569 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:41.237681 29569 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:41.240006 29569 zones.go:93] ListHostedZonesPages { } I0908 14:17:41.242332 29569 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 3.121s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.079s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.131s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.048s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.045s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0908 14:16:50.904795 15596 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0908 14:16:50.905039 15596 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.042s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0908 14:17:26.705209 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705468 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705517 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705556 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705597 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0908 14:17:26.705871 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705919 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705952 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.705986 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706025 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706057 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706096 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706138 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706177 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706214 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706246 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706283 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706316 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706349 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706382 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706431 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706466 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706498 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0908 14:17:26.706538 28686 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0908 14:17:26.706733 28686 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0908 14:17:26.706809 28686 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.092s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0908 14:17:07.314306 23458 builder.go:215] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.96s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0908 14:17:07.315264 23458 builder.go:215] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0908 14:17:07.315947 23458 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0908 14:17:07.316619 23458 builder.go:215] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0908 14:17:07.317172 23458 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0908 14:17:07.317898 23458 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0908 14:17:07.318462 23458 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0908 14:17:07.319152 23458 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0908 14:17:07.319168 23458 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0908 14:17:07.319179 23458 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0908 14:17:07.319189 23458 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 1.007s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.076s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.076s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.080s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.082s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.060s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.057s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.073s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.04s) PASS ok k8s.io/kops/pkg/edit 0.102s === RUN TestFlagToFalse I0908 14:16:30.786760 4741 featureflag_test.go:33] Created flag Unittest1 I0908 14:16:30.786889 4741 featureflag.go:162] FeatureFlag "UnitTest1"=false I0908 14:16:30.786907 4741 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0908 14:16:30.786988 4741 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.076s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.038s === RUN TestRenderInstanceGroupZones W0908 14:17:06.255760 23411 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.073s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0908 14:17:47.738650 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.738783 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.739997 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.740025 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.745245 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.745267 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.745278 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.745285 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.747110 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.747787 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.747816 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.749008 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.749025 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.754264 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.754320 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:47.754948 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:47.754977 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.756149 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.756171 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.761413 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.761463 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.761485 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.761499 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.762503 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.763296 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.763333 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.764531 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.764569 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.769896 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.769936 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:47.770539 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:47.770564 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.771809 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.771855 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.777159 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.777197 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:47.777898 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:47.777927 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.779132 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.779179 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.784464 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.784497 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.784514 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.784527 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:47.785376 29697 instancegroups.go:400] Draining the node: "node-2a.local". I0908 14:17:47.786121 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:47.786152 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.787369 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.787422 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.792745 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.792791 29697 instancegroups.go:400] Draining the node: "node-2b.local". I0908 14:17:47.793562 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:47.793596 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.794798 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.794852 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.800157 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.800207 29697 instancegroups.go:400] Draining the node: "node-2c.local". I0908 14:17:47.801056 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:47.801093 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.802309 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.802344 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.807613 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.807669 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0908 14:17:47.808148 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.808182 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.809377 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.809449 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.809488 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.809503 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.809542 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.810728 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.810780 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.810795 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:47.810824 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.812033 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.812068 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.812115 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.812130 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.812155 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.813367 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.813417 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.813438 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:47.813465 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.814692 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.814743 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.814757 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:47.814785 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.815962 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.815989 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.816037 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.816052 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:47.816078 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.817305 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.817364 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.817380 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:47.817407 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.818612 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0908 14:17:47.818673 29697 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0908 14:17:47.818689 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:47.818717 29697 instancegroups.go:436] waiting for 1ms after terminating instance W0908 14:17:47.819852 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0908 14:17:47.819886 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0908 14:17:47.820200 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.820234 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.821425 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.821471 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.821493 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.821548 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.821572 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.821587 29697 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.821603 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.822395 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.823142 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.823179 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.824299 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.824336 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.824356 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.824393 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:47.825322 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:47.825361 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.826520 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.826565 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.826610 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.826633 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.826654 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.826671 29697 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.826686 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.827921 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.828778 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.828813 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.829935 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.829971 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.829988 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.830025 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:47.830851 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:47.830886 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.832049 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.832093 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.832115 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.832157 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:47.833006 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:47.833043 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.834160 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.834190 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.834207 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.834222 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.834241 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.834255 29697 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.834267 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:47.835229 29697 instancegroups.go:400] Draining the node: "node-2a.local". I0908 14:17:47.836013 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:47.836047 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.837217 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.837262 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.837301 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.837346 29697 instancegroups.go:400] Draining the node: "node-2b.local". I0908 14:17:47.838180 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:47.838220 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.839340 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.839375 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.839392 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.839429 29697 instancegroups.go:400] Draining the node: "node-2c.local". I0908 14:17:47.840198 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:47.840236 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.841422 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.841467 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0908 14:17:47.841489 29697 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0908 14:17:47.841507 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0908 14:17:47.841784 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0908 14:17:47.842026 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.842057 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.843174 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.843233 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.848449 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.848509 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.848528 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.848541 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.849370 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.850255 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.850290 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.851409 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.851438 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.856598 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.856644 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:47.857617 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:47.857652 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.858846 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.858885 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.864120 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.864163 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.864202 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.864217 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.865346 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.866243 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.866278 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.867468 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.867522 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.872770 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.872826 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:47.873813 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:47.873850 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.875060 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.875099 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.880349 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.880403 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:47.881378 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:47.881415 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.882613 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.882652 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.887913 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.887946 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.887965 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.887979 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:47.889091 29697 instancegroups.go:400] Draining the node: "node-2a.local". I0908 14:17:47.890004 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:47.890039 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.891235 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.891274 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.896514 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.896568 29697 instancegroups.go:400] Draining the node: "node-2b.local". I0908 14:17:47.897523 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:47.897561 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.898759 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.898798 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.904048 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.904099 29697 instancegroups.go:400] Draining the node: "node-2c.local". I0908 14:17:47.905038 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:47.905074 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.906270 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.906309 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.911566 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.911603 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0908 14:17:47.911726 29697 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0908 14:17:47.912259 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.912292 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.913491 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.913536 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.913556 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0908 14:17:47.913920 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.913980 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.915184 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.915224 29697 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0908 14:17:47.915266 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0908 14:17:47.915556 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.915586 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.915602 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0908 14:17:47.915779 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.915805 29697 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0908 14:17:47.915821 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0908 14:17:47.916076 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.916118 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.917258 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.917320 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.922576 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.922647 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.922672 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.922688 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.923546 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.924437 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.924475 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.925676 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.925739 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.925761 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0908 14:17:47.926163 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.926196 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.927384 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.927425 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.932672 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.932742 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.932768 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.932801 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.933661 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.934630 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.934666 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.935862 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.935909 29697 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0908 14:17:47.935931 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0908 14:17:47.936278 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.936307 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.936320 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.937489 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.941216 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.941279 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.942507 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.942549 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.947731 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.947842 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:47.948791 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:47.948838 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.950069 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.950105 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.955434 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.955484 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:47.956298 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:47.956334 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.957516 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.957552 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.963178 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.963238 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.03s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0908 14:17:47.963661 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.963694 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.963713 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0908 14:17:47.964002 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.964033 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.964057 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0908 14:17:47.964341 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.964369 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.964385 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0908 14:17:47.964579 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.964608 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.964622 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.965743 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.966573 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.966610 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.967804 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.967852 29697 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0908 14:17:47.967872 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0908 14:17:47.968123 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.968159 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.968173 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:47.969276 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:47.970100 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:47.970135 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.971321 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.971363 29697 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0908 14:17:47.971385 29697 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0908 14:17:47.971693 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:47.971724 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.972931 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.972978 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:47.974144 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.979381 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:47.980554 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.985800 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:47.986988 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:47.992278 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.992367 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.992398 29697 instancegroups.go:503] Cluster validated. I0908 14:17:47.992416 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:47.993205 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:47.994130 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:47.994168 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:47.995377 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:47.995418 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.000682 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.000768 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:48.002100 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:48.002172 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.003503 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.003559 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.008851 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.008892 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.008919 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.008934 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:48.010001 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.011251 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.011291 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.012500 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.012554 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.017809 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.017868 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.018798 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.018836 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.020032 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.020082 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.025339 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.025407 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:48.026243 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:48.026282 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.027578 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.027628 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.032894 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.032927 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.032955 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.032983 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:48.034067 29697 instancegroups.go:400] Draining the node: "node-2a.local". I0908 14:17:48.034886 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:48.034925 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.036110 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.036159 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.041421 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.041481 29697 instancegroups.go:400] Draining the node: "node-2b.local". I0908 14:17:48.042375 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:48.042415 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.043624 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.043672 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.048913 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.048975 29697 instancegroups.go:400] Draining the node: "node-2c.local". I0908 14:17:48.049798 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:48.049837 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.051021 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.051071 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.056375 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.056410 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0908 14:17:48.056742 29697 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0908 14:17:48.056776 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.057964 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.058044 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:48.059189 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:48.060382 29697 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0908 14:17:48.061560 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.066850 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.066926 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.066960 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.066975 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:48.068141 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:48.069258 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:48.069296 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.070502 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.070544 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.075833 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.075885 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:48.076716 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:48.076753 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.077956 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.077992 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.083287 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.083327 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.083349 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.083364 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:48.084404 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.085159 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.085194 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.086414 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.086449 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.091720 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.091773 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.092622 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.092663 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.093854 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.093890 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.099168 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.099221 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:48.100033 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:48.100070 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.101285 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.101334 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.106626 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.106659 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.106680 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.106695 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:48.107703 29697 instancegroups.go:400] Draining the node: "node-2a.local". I0908 14:17:48.108493 29697 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0908 14:17:48.108529 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.109747 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.109783 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.115083 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.115136 29697 instancegroups.go:400] Draining the node: "node-2b.local". I0908 14:17:48.116031 29697 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0908 14:17:48.116068 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.117261 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.117322 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.122578 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.122633 29697 instancegroups.go:400] Draining the node: "node-2c.local". I0908 14:17:48.123492 29697 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0908 14:17:48.123529 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.124704 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.124738 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.129981 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.130016 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0908 14:17:48.130654 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.130682 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.130695 29697 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0908 14:17:48.131456 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.132214 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.132249 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.133454 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.133498 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.138809 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.138865 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.139687 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.139722 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.140895 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.140942 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.146226 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.146259 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0908 14:17:48.146492 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.146521 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.146537 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:48.147314 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:48.148100 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:48.148135 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.149332 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.149366 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.154607 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.154658 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:48.155587 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:48.155623 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.156821 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.156855 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.162077 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.162109 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0908 14:17:48.162441 29697 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled W0908 14:17:48.162491 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0908 14:17:48.162513 29697 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled W0908 14:17:48.162528 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0908 14:17:48.162548 29697 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled W0908 14:17:48.162558 29697 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0908 14:17:48.162576 29697 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0908 14:17:48.162589 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0908 14:17:48.162855 29697 instancegroups.go:570] Detaching instance "bastion-1a", in group "bastion-1". I0908 14:17:48.162911 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:48.164092 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.164127 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.169649 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.169722 29697 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled I0908 14:17:48.169822 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.169866 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.169892 29697 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0908 14:17:48.170924 29697 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled I0908 14:17:48.170969 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.171003 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.171029 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:48.171964 29697 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0908 14:17:48.172061 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:48.173314 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.173349 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.178609 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.178659 29697 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled I0908 14:17:48.178685 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.178717 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.178741 29697 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0908 14:17:48.179711 29697 instancegroups.go:568] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0908 14:17:48.179798 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:48.181017 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.181053 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:48.186302 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.186354 29697 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0908 14:17:48.186382 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0908 14:17:48.186761 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.186813 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.186841 29697 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0908 14:17:48.189060 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.189946 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.190013 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.191252 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.191320 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.191528 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.191906 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:48.193266 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.193567 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:48.193610 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.194820 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.194902 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.295398 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.295464 29697 instancegroups.go:400] Draining the node: "node-1e.local". I0908 14:17:48.295527 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:48.296890 29697 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0908 14:17:48.297317 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:48.297368 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.298588 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.298634 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.298676 29697 instancegroups.go:400] Draining the node: "node-1f.local". I0908 14:17:48.299610 29697 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0908 14:17:48.299657 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.300876 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.300952 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.402313 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.402423 29697 instancegroups.go:400] Draining the node: "node-1g.local". I0908 14:17:48.403521 29697 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0908 14:17:48.403576 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.404798 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.404857 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.404877 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0908 14:17:48.405248 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.405286 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.405301 29697 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0908 14:17:48.407344 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.407506 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.408528 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.408847 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.408893 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.410141 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.410210 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.511071 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.511140 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:48.511176 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:48.514553 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:48.514837 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:48.514887 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.516112 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.516162 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.516212 29697 instancegroups.go:400] Draining the node: "node-1e.local". I0908 14:17:48.517090 29697 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0908 14:17:48.517135 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.518331 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.518408 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.618628 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.618677 29697 instancegroups.go:400] Draining the node: "node-1f.local". I0908 14:17:48.619527 29697 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0908 14:17:48.619572 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.620761 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.620808 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.620827 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0908 14:17:48.621077 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.621110 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.621125 29697 instancegroups.go:311] Tainting 7 nodes in "master-1" instancegroup. I0908 14:17:48.623383 29697 instancegroups.go:400] Draining the node: "master-1a.local". I0908 14:17:48.624168 29697 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0908 14:17:48.624214 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.625419 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.625469 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.625511 29697 instancegroups.go:400] Draining the node: "master-1c.local". I0908 14:17:48.625593 29697 instancegroups.go:400] Draining the node: "master-1b.local". I0908 14:17:48.626828 29697 instancegroups.go:591] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0908 14:17:48.627109 29697 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0908 14:17:48.627155 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.628350 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.628425 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.729033 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.729093 29697 instancegroups.go:400] Draining the node: "master-1e.local". I0908 14:17:48.729163 29697 instancegroups.go:400] Draining the node: "master-1d.local". I0908 14:17:48.730502 29697 instancegroups.go:591] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0908 14:17:48.730842 29697 instancegroups.go:591] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0908 14:17:48.730890 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.732136 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.732180 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.732223 29697 instancegroups.go:400] Draining the node: "master-1f.local". I0908 14:17:48.733076 29697 instancegroups.go:591] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0908 14:17:48.733121 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.734325 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.734401 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.835015 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.835071 29697 instancegroups.go:400] Draining the node: "master-1g.local". I0908 14:17:48.835913 29697 instancegroups.go:591] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0908 14:17:48.835959 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.837194 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.837237 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.837256 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0908 14:17:48.837516 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.837554 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.837571 29697 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0908 14:17:48.839402 29697 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0908 14:17:48.839473 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:48.840670 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.840715 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.840731 29697 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0908 14:17:48.840792 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:48.842009 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.842053 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.842108 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:48.842174 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:48.843510 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:48.843821 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:48.843866 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.845059 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.845134 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.945769 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.945821 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:48.945904 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:48.947210 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:48.947571 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:48.947621 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.948843 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.948887 29697 instancegroups.go:503] Cluster validated. I0908 14:17:48.948929 29697 instancegroups.go:400] Draining the node: "node-1e.local". I0908 14:17:48.949755 29697 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0908 14:17:48.949805 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:48.950989 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:48.951065 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.051535 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.051599 29697 instancegroups.go:400] Draining the node: "node-1f.local". I0908 14:17:49.052495 29697 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0908 14:17:49.052545 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.053770 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.053813 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.053830 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0908 14:17:49.054069 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.054105 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.054119 29697 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0908 14:17:49.055889 29697 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0908 14:17:49.055936 29697 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0908 14:17:49.055982 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:49.057201 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.057245 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.057311 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.057355 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.058676 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.059016 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.059068 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.060282 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.060351 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.160869 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.160926 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:49.161013 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:49.162245 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:49.162581 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:49.162635 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.163873 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.163916 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.164001 29697 instancegroups.go:400] Draining the node: "node-1e.local". I0908 14:17:49.164843 29697 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0908 14:17:49.164893 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.166131 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.166206 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.267095 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.267162 29697 instancegroups.go:400] Draining the node: "node-1f.local". I0908 14:17:49.268008 29697 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0908 14:17:49.268058 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.269281 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.269323 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.269341 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0908 14:17:49.269542 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.269569 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.269582 29697 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0908 14:17:49.270271 29697 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0908 14:17:49.270302 29697 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0908 14:17:49.270340 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:49.271571 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.271606 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.276885 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.276938 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.276958 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.278290 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.278331 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.278599 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.278632 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.279568 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.279605 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.284927 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.284961 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0908 14:17:49.285188 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.285217 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.285230 29697 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0908 14:17:49.285917 29697 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0908 14:17:49.285980 29697 instancegroups.go:160] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0908 14:17:49.285993 29697 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0908 14:17:49.286032 29697 instancegroups.go:160] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0908 14:17:49.286066 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:49.287298 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.287346 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.292653 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.292708 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.292743 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.294085 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.294126 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.294398 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.294432 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.295340 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.295374 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.300816 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.300850 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0908 14:17:49.301068 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.301103 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.301117 29697 instancegroups.go:311] Tainting 4 nodes in "node-1" instancegroup. I0908 14:17:49.302350 29697 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0908 14:17:49.302413 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:49.303623 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.303666 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.303680 29697 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0908 14:17:49.303733 29697 instancegroups.go:172] waiting for 1ms after detaching instance I0908 14:17:49.304905 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.304948 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.304994 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:49.305064 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.305133 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.306761 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:49.306813 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.307110 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.307168 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.307583 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.307638 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.308000 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.408635 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.408694 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:49.409597 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:49.409644 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.410866 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.410910 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.410934 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0908 14:17:49.411139 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.411171 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.411188 29697 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0908 14:17:49.413332 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.414090 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.414135 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.415341 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.415390 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.415434 29697 instancegroups.go:400] Draining the node: "node-1e.local". I0908 14:17:49.415497 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:49.416788 29697 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0908 14:17:49.417104 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:49.417192 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.418437 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.418507 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.519317 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.519370 29697 instancegroups.go:400] Draining the node: "node-1g.local". I0908 14:17:49.519394 29697 instancegroups.go:400] Draining the node: "node-1f.local". I0908 14:17:49.520745 29697 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0908 14:17:49.521076 29697 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0908 14:17:49.521127 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.522357 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.522399 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.522440 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.523264 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.523308 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.524495 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.524571 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.625203 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.625253 29697 instancegroups.go:400] Draining the node: "node-1d.local". I0908 14:17:49.626072 29697 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0908 14:17:49.626116 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.627441 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.627485 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.627502 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0908 14:17:49.627720 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.627749 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.627769 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0908 14:17:49.627793 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0908 14:17:49.627812 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0908 14:17:49.627829 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0908 14:17:49.627983 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.628009 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.628021 29697 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0908 14:17:49.629068 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0908 14:17:49.629105 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0908 14:17:49.629125 29697 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0908 14:17:49.629170 29697 instancegroups.go:400] Draining the node: "node-1a.local". I0908 14:17:49.629930 29697 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0908 14:17:49.629964 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.631161 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.631195 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.636504 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.636568 29697 instancegroups.go:400] Draining the node: "node-1b.local". I0908 14:17:49.637429 29697 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0908 14:17:49.637467 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.638689 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.638723 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.644010 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.644061 29697 instancegroups.go:400] Draining the node: "node-1c.local". I0908 14:17:49.644892 29697 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0908 14:17:49.644929 29697 instancegroups.go:436] waiting for 1ms after terminating instance I0908 14:17:49.646144 29697 instancegroups.go:470] Validating the cluster. I0908 14:17:49.646178 29697 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0908 14:17:49.651461 29697 instancegroups.go:503] Cluster validated. I0908 14:17:49.651501 29697 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.988s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.040s === RUN TestParse E0908 14:16:50.880781 15578 versions.go:96] unable to parse Kubernetes version "" E0908 14:16:50.881038 15578 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.049s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.028s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.18s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.01s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.254s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.26s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.330s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.081s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.072s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0908 14:17:21.502130 27709 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.072s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I0908 14:17:28.172532 28908 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0908 14:17:28.339812 28908 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0908 14:17:28.340063 28908 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0908 14:17:28.347627 28908 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0908 14:17:28.347804 28908 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.18s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.07s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.08s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.02s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.248s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I0908 14:17:27.344518 28835 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.13s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.13s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.225s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.075s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.01s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.086s === RUN TestServerGroupBuilder I0908 14:17:28.776803 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.795045 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.809831 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.827989 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.878019 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.900347 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.914866 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.925509 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:28.978926 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:29.022554 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:29.029809 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:29.063952 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:29.072693 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:29.083672 29094 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.31s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.398s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.031s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.008s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.036s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.97s) --- PASS: TestIssueCert/ca (0.83s) --- PASS: TestIssueCert/client (0.15s) --- PASS: TestIssueCert/clientOneYear (1.00s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.996s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.054s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I0908 14:17:26.236423 28572 routetable.go:53] DescribeRouteTables: { } I0908 14:17:26.238006 28572 aws.go:225] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0908 14:17:26.238102 28572 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0908 14:17:26.238421 28572 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0908 14:17:26.238637 28572 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0908 14:17:26.238944 28572 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0908 14:17:26.239158 28572 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0908 14:17:26.239262 28572 volumes.go:128] DescribeVolumes: { } W0908 14:17:26.239340 28572 aws.go:2239] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0908 14:17:26.239766 28572 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:26.240148 28572 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:26.240765 28572 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:26.240914 28572 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:26.241033 28572 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:26.241129 28572 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:26.241190 28572 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:26.241271 28572 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:26.241348 28572 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:26.241417 28572 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0908 14:17:26.241502 28572 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0908 14:17:26.241578 28572 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0908 14:17:26.241675 28572 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0908 14:17:26.241757 28572 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0908 14:17:26.241890 28572 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0908 14:17:26.241971 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0908 14:17:26.242041 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0908 14:17:26.242104 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0908 14:17:26.242166 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0908 14:17:26.242231 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0908 14:17:26.242293 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0908 14:17:26.242360 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0908 14:17:26.242419 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0908 14:17:26.242505 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0908 14:17:26.242571 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0908 14:17:26.242673 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0908 14:17:26.242756 28572 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.147s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.027s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.070s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.006s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.007s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.072s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.073s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.011s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.058s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.077s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.007s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0908 14:16:31.219763 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220589 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220604 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220632 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220646 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220659 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220671 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220684 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220696 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220709 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220722 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220734 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220747 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220759 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.220771 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.220784 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223258 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223294 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223392 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223412 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223503 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223523 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223612 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223632 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223720 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223739 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223842 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223862 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.223965 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.223984 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224103 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224120 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224200 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224218 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224303 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224322 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224408 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224427 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224534 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224553 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224637 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224655 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224741 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224760 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224847 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224865 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.224966 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.224985 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225075 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225094 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225180 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225199 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225285 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225315 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225450 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225468 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225542 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225559 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225640 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225660 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225734 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225750 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225830 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225848 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.225918 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.225934 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226009 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226026 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226120 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226137 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226208 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226224 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226296 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226343 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226426 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226444 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226524 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226546 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226661 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226680 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226774 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226792 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226875 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226894 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.226973 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.226989 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227063 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227079 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227177 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227195 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227302 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227319 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227398 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227413 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227495 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227513 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227637 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227654 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227728 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227744 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227837 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227854 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.227932 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.227950 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228026 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228044 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228120 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228138 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228231 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228249 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228334 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228352 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228433 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228452 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228531 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228547 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228627 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228646 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228748 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228777 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228855 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228872 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.228947 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.228967 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.229047 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.229066 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.229159 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.229177 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.229253 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.242659 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.242797 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.242825 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.242917 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.242942 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243025 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243048 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243128 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243150 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243230 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243253 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243382 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243408 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243497 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243521 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243606 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243630 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243711 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243735 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243812 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243834 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.243915 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.243938 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244048 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244070 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244148 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244169 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244240 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244258 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244330 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244348 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244421 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244439 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244512 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244558 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244639 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244657 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244729 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244748 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244819 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244839 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.244935 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.244957 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245037 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245057 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245139 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245159 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245234 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245253 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245329 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245349 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245425 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245445 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245562 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245583 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245667 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245688 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245770 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245794 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245874 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245895 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.245978 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.245998 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246076 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246097 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246176 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246198 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246278 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246298 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246379 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246399 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246483 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246503 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246588 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246609 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246685 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246704 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246799 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246820 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.246907 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.246928 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.247005 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.247025 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.247101 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.247119 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0908 14:16:31.248041 5280 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0908 14:16:31.248061 5280 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0908 14:16:31.248076 5280 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0908 14:16:31.248090 5280 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0908 14:16:31.248103 5280 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0908 14:16:31.248117 5280 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0908 14:16:31.248129 5280 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0908 14:16:31.248164 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248184 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248198 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.248211 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248225 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.248247 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248257 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.248265 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248272 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.248279 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248286 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248297 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248308 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248319 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.248327 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.248334 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.258779 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.258821 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.258912 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.258938 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.259022 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.259045 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.259137 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.259163 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.259327 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.259353 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.259774 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.259799 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.259882 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.259905 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260003 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260022 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260103 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260127 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260209 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260228 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260310 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260330 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260420 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260442 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260538 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260558 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260653 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260672 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260759 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260777 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260861 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260880 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.260962 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.260981 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261065 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261084 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261173 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261191 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261279 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261298 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261381 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261399 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261479 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261499 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261597 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261616 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261705 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261724 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261814 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261831 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.261964 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.261984 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262071 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262090 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262182 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262201 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262320 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262355 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262449 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262469 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262553 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262571 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262654 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262672 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262754 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262772 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262853 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262871 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.262951 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.262968 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263059 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263078 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263165 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263183 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263268 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263286 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263375 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263395 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263481 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263503 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263586 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263605 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263687 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263714 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263801 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263818 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.263907 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.263925 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264060 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264080 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264285 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264305 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264390 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264408 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264496 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264516 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264599 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264616 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264696 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264715 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264801 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264823 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.264932 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.264951 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265059 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265079 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265158 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265176 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265262 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265281 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265364 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265384 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265465 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265492 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265575 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265599 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265689 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265714 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265804 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265829 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.265914 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.265942 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266031 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266054 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266138 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266163 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266251 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266275 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266358 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266382 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266470 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266495 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266586 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266610 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266701 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266725 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266825 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266850 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.266934 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.266958 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267038 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267061 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267141 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267164 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267240 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267262 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267359 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267383 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267473 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267497 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267605 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267630 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267750 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267773 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267853 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267874 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.267949 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.267969 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268045 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268069 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268147 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268167 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268247 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268267 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268350 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268369 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268448 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268471 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268551 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268573 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268659 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268680 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268758 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268779 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268862 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268883 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.268962 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.268982 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269060 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269082 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269160 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269180 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269264 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269286 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269365 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269389 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269466 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269486 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269560 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269580 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269655 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269676 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269748 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269767 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269847 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269867 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0908 14:16:31.269941 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0908 14:16:31.269961 5280 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.073s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.042s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.074s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.02s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) PASS ok k8s.io/kops/tests/integration/conversion 0.037s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0908 14:17:08.043685 23892 files.go:57] Writing file "/tmp/TestWriteFile2294284529/001/SubDir/test1.tmp" I0908 14:17:08.043886 23892 files.go:113] Changing file mode for "/tmp/TestWriteFile2294284529/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.027s === RUN TestBootstrapChannelBuilder_BuildTasks I0908 14:17:34.392130 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:34.392475 29476 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:34.392548 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:34.392598 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:34.392658 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:34.392693 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:34.392711 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:34.392737 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:34.392761 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:34.407235 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.462927 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.581062 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.653392 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.716375 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.780572 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.836038 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:34.897625 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:35.029236 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:36.172557 29476 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (2.50s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0908 14:17:36.890630 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:36.890900 29476 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:36.891026 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:36.891089 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:36.891146 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:36.891193 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:36.891224 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:36.891266 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:36.891297 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:36.891332 29476 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0908 14:17:36.898244 29476 zones.go:93] ListHostedZonesPages { } I0908 14:17:37.005820 29476 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.12s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0908 14:17:37.006013 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0908 14:17:37.006152 29476 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:37.006208 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0908 14:17:37.006239 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0908 14:17:37.006268 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0908 14:17:37.006295 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0908 14:17:37.006311 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0908 14:17:37.006333 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0908 14:17:37.006349 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0908 14:17:37.066770 29476 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.17s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.05s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.13s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0908 14:17:37.376841 29476 new_cluster.go:980] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_Image_Required I0908 14:17:37.406151 29476 populate_instancegroup_spec.go:323] Cannot set default Image for CloudProvider="aws" --- PASS: TestPopulateInstanceGroup_Image_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0908 14:17:37.406813 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.406835 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.406846 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0908 14:17:37.406865 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0908 14:17:37.406876 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0908 14:17:37.406894 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0908 14:17:37.406913 29476 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0908 14:17:37.406930 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.406942 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0908 14:17:37.406957 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.406968 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0908 14:17:37.406985 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.407010 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.407030 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0908 14:17:37.407040 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0908 14:17:37.407049 29476 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0908 14:17:37.407058 29476 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0908 14:17:37.407069 29476 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0908 14:17:37.407079 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0908 14:17:37.407090 29476 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0908 14:17:37.407100 29476 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0908 14:17:37.408058 29476 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 3.100s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0908 14:17:23.156340 27877 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2315262659/001 I0908 14:17:23.159255 27877 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2315262659/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestAutoscalingGroupCloudformationRender I0908 14:17:23.161028 27877 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender985287820/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0908 14:17:23.162183 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0908 14:17:23.162294 27877 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0908 14:17:23.162428 27877 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0908 14:17:23.162769 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.162944 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.163058 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.163095 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.163395 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.163458 27877 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0908 14:17:23.163525 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.163631 27877 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0908 14:17:23.163750 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0908 14:17:23.163971 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.164050 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.164118 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.164148 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.164201 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.164264 27877 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0908 14:17:23.164371 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0908 14:17:23.164812 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.164883 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.165034 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.165111 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.165152 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.165194 27877 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0908 14:17:23.165307 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.165588 27877 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0908 14:17:23.165657 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.165700 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.165740 27877 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0908 14:17:23.165837 27877 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.166108 27877 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0908 14:17:23.166199 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.166242 27877 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0908 14:17:23.166307 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0908 14:17:23.166519 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.166589 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.166662 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.166692 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.166751 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.166795 27877 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0908 14:17:23.166856 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.166948 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.166990 27877 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0908 14:17:23.167052 27877 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.167157 27877 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0908 14:17:23.167213 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0908 14:17:23.167290 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0908 14:17:23.167518 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0908 14:17:23.167584 27877 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0908 14:17:23.167627 27877 internetgateways.go:66] CreateInternetGateway: { } I0908 14:17:23.167672 27877 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0908 14:17:23.167860 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.167926 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.168015 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.168047 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.168159 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.168202 27877 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0908 14:17:23.168260 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.168373 27877 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0908 14:17:23.168489 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0908 14:17:23.168668 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.168733 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.168799 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.168829 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.168882 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.168940 27877 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0908 14:17:23.169030 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0908 14:17:23.170399 27877 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender3811171662/001 I0908 14:17:23.170812 27877 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender3811171662/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0908 14:17:23.174189 27877 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender994152435/001 I0908 14:17:23.176662 27877 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender994152435/001 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0908 14:17:23.177659 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.177750 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.177921 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.178000 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.178045 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.178088 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.178186 27877 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0908 14:17:23.178508 27877 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0908 14:17:23.178577 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0908 14:17:23.178629 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0908 14:17:23.178822 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.178890 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.178965 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.178995 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.179041 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.179084 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.179144 27877 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0908 14:17:23.179260 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0908 14:17:23.179796 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.179863 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.179999 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.180068 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.180108 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.180148 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.180214 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.180376 27877 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0908 14:17:23.180442 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.180483 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.180525 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0908 14:17:23.180724 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.180793 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.180865 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.180893 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.180935 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.180976 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.181040 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.181129 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.181171 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0908 14:17:23.181578 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.181648 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.181784 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.181844 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.181880 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.181936 27877 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0908 14:17:23.182011 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.182154 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.182219 27877 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0908 14:17:23.182311 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.182366 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.182456 27877 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:17:23.182507 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0908 14:17:23.182692 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.182739 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.182786 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.182802 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.182827 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.182849 27877 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0908 14:17:23.182890 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.182955 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.182977 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.183024 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0908 14:17:23.183301 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.183343 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.183433 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.183472 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.183495 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.183517 27877 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0908 14:17:23.183561 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.183680 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.183730 27877 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0908 14:17:23.183795 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.183821 27877 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0908 14:17:23.183872 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.183914 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0908 14:17:23.184156 27877 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0908 14:17:23.184228 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.184301 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.184333 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.184376 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.184417 27877 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0908 14:17:23.184493 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.184548 27877 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0908 14:17:23.184631 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.184658 27877 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0908 14:17:23.184800 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0908 14:17:23.184863 27877 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0908 14:17:23.185027 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.185069 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.185130 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.185148 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.185220 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.185249 27877 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0908 14:17:23.185297 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.185332 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.185427 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0908 14:17:23.185453 27877 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0908 14:17:23.185482 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0908 14:17:23.185609 27877 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0908 14:17:23.185650 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.185687 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.185701 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.185731 27877 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0908 14:17:23.185763 27877 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0908 14:17:23.185830 27877 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0908 14:17:23.186060 27877 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:23.186104 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.186198 27877 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0908 14:17:23.186241 27877 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0908 14:17:23.186263 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.186285 27877 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:23.186410 27877 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:23.186451 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0908 14:17:23.186491 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.186506 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.186535 27877 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0908 14:17:23.186560 27877 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0908 14:17:23.186730 27877 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0908 14:17:23.186865 27877 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:23.186907 27877 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0908 14:17:23.186948 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0908 14:17:23.186963 27877 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0908 14:17:23.186992 27877 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.085s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.053s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.024s === RUN TestDiskRenderAzure I0908 14:17:13.137187 25318 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0908 14:17:13.137613 25318 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0908 14:17:13.137837 25318 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0908 14:17:13.138268 25318 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0908 14:17:13.138474 25318 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0908 14:17:13.138726 25318 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0908 14:17:13.138965 25318 resourcegroup.go:103] Creating a new Resource Group with name: rg I0908 14:17:13.138978 25318 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0908 14:17:13.139148 25318 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0908 14:17:13.139655 25318 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0908 14:17:13.139907 25318 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0908 14:17:13.140154 25318 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0908 14:17:13.140474 25318 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0908 14:17:13.140708 25318 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.034s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.043s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I0908 14:17:22.217160 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.218143 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.218334 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.218986 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.219180 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.219411 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0908 14:17:22.219764 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.219911 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.220090 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.220290 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.220472 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.220567 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0908 14:17:22.220907 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.221320 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.221513 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.222125 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0908 14:17:22.222359 27803 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0908 14:17:22.222601 27803 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.074s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.035s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.029s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.026s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.017s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.025s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0908 14:17:32.626253 29418 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0908 14:17:32.626392 29418 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0908 14:17:32.626405 29418 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0908 14:17:32.626417 29418 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0908 14:17:32.626427 29418 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0908 14:17:32.626437 29418 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0908 14:17:32.626447 29418 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0908 14:17:32.626927 29418 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.078s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.027s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.004s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.004s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.018s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0908 14:17:07.298687 23773 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.018s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.016s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.003s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.004s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.020s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0908 14:17:22.470504 27842 s3context.go:90] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0908 14:17:22.470605 27842 s3context.go:238] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.060s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 124.4 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.24.1-r1.apk >>> kops: Build complete at Thu, 08 Sep 2022 14:18:41 +0000 elapsed time 0h 4m 13s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/2) Purging .makedepends-kops (20220908.141429) (2/2) Purging kubernetes (1.25.0-r1) OK: 807 MiB in 104 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...