1.6.0

kops 1.6.0 adds support for kubernetes 1.6.x

Features and Bug Fixes

• Only run multiple masters if master-count or multiple master-zones are specified

• Don't run dns-controller with ingress enabled; caused issues in existing setups

• Expose apiserver audit log by setting it as a Host Path (thanks @ottoyiu)

• Add api-loadbalancer option to cli set public or internal loadbalancer (thanks @mikesplain)

• Update the terraform generator to use the value "role" instead of "roles" for the aws_iam_instance_profile resource (thanks @while1eq1)

• Add required terraform version declaration & require terraform 0.9.3

• Add instance role names/ARNs to terraform output (thanks @pastjean)

• Recognize networking=kopeio as alias for kopeio-vxlan (thanks @rdtr)

• Add critical pod annotations to kopeio networking

• Creates .kube dir at /home/admin with group/owner as admin (thanks @dolftax)

• Remove babysit-daemons flag from 1.7

• Fix channel version recommendations

• Add missing error handling (thanks @caarlos0)

• Update calico to 2.1.5 (thanks @blakebarnett)

• Add ingress nginx example addons for 1.6

• Add heapster addon for 1.6 (thanks @sethpollack, @BradErz)

• Update alpha channel

• Implemented a Git hook for the CI tests and related installer (thanks @WillemMali)

• Ensure state store not set when generating docs (thanks @rdtr)

• CI now verifies that docs have been generated (thanks @chrislovecnm)

• Added help target to Makefile and fixed small path ordering issue (thanks @WillemMali)

• Mark all phony targets with ".PHONY" in Makefile (thanks @WillemMali)

• Adds instructions for developing in Docker (thanks @svozza)

• Improved documentation & polish (thanks @BradErz, @dolftax, @four43, @lcrisci, @pierreozoux, @sethpollack, @sngchlko, @snoby, @svozza)

• Review of all command docs / help (thanks @gianrubio, @chrislovecnm)

Known Issues

• Kubernetes upgrade from 1.5 -> 1.6 requires a configmap to be created in the kube-system namespace. If kubectl -n kube-system get configmap does not return a configmap with the name "kube-dns". BEFORE upgrade please execute kubectl create configmap -n kube-system kube-dns.
• Kubernetes upgrades from 1.5 -> 1.6 users have noticed some issues with taints when running HA masters. Editing the taints by hand is a workaround. See: #2594
• Some Kubernetes upgrades from 1.6.x -> 1.6.x have experience issues with secrets and service accounts. Deleting the secrets have been a workaround (not validated). See: #2576
• Calico users have noticed problems using Calico with Kubernetes 1.6, this has been fixed in master, and will be released in kops 1.6.1.

1.6.0-beta.1

Deploys dns-controller with --watch-ingress=true (thanks @geojaz)
Dockerfile for running kops (thanks @denniswebb)
zsh autocompletion (thanks @chrislovecnm)
Add support for m4.16xlarge instance types (thanks @ryanlitalien)
Set default master size for ap-northeast-2 (thanks @buo)
kops create cluster flag for encrypted etcd volumes (thanks @pronix)
Enable CertificateSigner API on k8s 1.6
Fix validation when we have fewer nodes than minimum count (thanks @GauntletWizard)
Refactor cluster deletion (thanks @andrewsykim)
Misc fixes for golang vet warnings (thanks @pronix)
Use explicit tag management in network tasks
Fix private DNS zone creation

Experimental support for gossip-backed DNS
Initial vmware backend (thanks @abrarshivani, @luomiao, @prashima, @SandeepPissay, @vmware,
Initial EFK addon, and fix version in EFK manifest (thanks @archseer)
Experimental support for hooks

Update flannel & canal to 0.7.1, containing k8s 1.6 fixes
Update heapster to 1.3.0 and addon-resizer to 1.7 (thanks @blackstar257)
Update CNI version for k8s 1.6 (thanks @chrislovecnm)
Update route53-mapper addon (thanks @a1dutch, @LeandroCR)
Use go 1.8.1 (thanks @chrislovecnm)
Updated k8s dashboard to 1.6 (thanks @mstump)

Cleaned up CLI help and output (thanks @robertojrojas)
Clarified brew instructions around --HEAD (thanks @chrislovecnm)
Lots more polish & documentation (thanks @arthurlm44, @BrentDorsey, @caarlos0, @chrislovecnm, @geojaz, @pierreozoux, @snoby, @tazjin, @voyalab, @while1eq1)

1.6.0-alpha.2

Known issues:

• We are tracking a potential issue with flannel & calico which seems to cause the masters to lose their taints
• Please review the 1.6.0-alpha.1 notes

Major changes & fixes in alpha.2:

• Update calico, canal, flannel, kopeio-networking and weave for 1.6 (thanks @felipejfc, @ottoyiu, @pronix)
• Update cluster-autoscaler for 1.6 (thanks @ottoyiu)
• Support version selection in the kops addon manager
• Reuse the public key if it exists, so upgrade doesn't require rekeying
• Support private dns with terraform (thanks @ahl)
• Make sure we start docker-healthcheck.service
• Update to docker 1.12.6 for k8s 1.6
• Fix CNI bin & conf paths (thanks @prachetasp)
• AWS Instance Tenancy support(thanks @dacohen)
• Fix GCE disk cleanup (thanks @andrewsykim)

More goodies:

• Add AWS CloudConfig DisableSecurityGroupIngress Configuration Parameter (thanks @waldman)
• Update protokube's embedded kubectl to 1.6.1 (thanks @chrislovecnm)
• Implement --filename option for kops delete (thanks @MrTrustor)
• Update route53-mapper v.1.3.0 (thanks @a1dutch)
• Set nf_conntrack_max in kubeproxy (thanks @andrewsykim)
• Feature flags support for kubelet (thanks @chrislovecnm)
• Expose options.CreateKubecfg to the CLI (thanks @kris-nova)
• kopeio networking should not set configure-cloud-routes
• Fix node role logic in validate cluster command
• Docs & polish fixes (thanks @adrianmoisey, @caarlos0, @chrislovecnm, @kris-nova, @MrTrustor, @pronix, @twpayne )

1.6.0-alpha.1

1.6.0-alpha.1

1.6.0-alpha.1 is a prerelease early-access of kops 1.6, which is the release with full support for kubernetes 1.6.
This version of kops & kubernetes has not yet undergone extensive validation, and there will be improvements
made before release of kops 1.6.0.

This is not a full set of release notes, but rather a summary of the highest impact changes in the 1.6 release:

• RBAC can be enabled by passing the --authorization=rbac parameter to kops create cluster,
  or via kops edit cluster and change authorization from alwaysAllow: {} to rbac: {}

• The standard RBAC policy for 1.6 means that all access to the Kubernetes API using the default
  service account method will be denied.

• The taints & tolerations have changed as part of their graduation from alpha. The taint is now a field on the node:

spec:
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master

An example toleration (as used in dns-controller) is:

spec:
  tolerations:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master

Note that the annotation form is ignored. To schedule a pod on the master, the toleration must be updated
and moved from an annotation to the field.

• A new label for nodes, mirroring the toleration, is added and is now preferred: node-role.kubernetes.io/master=
  (node-role.kubernetes.io/master with an empty value). kubernetes.io/role=master is still present, but
  the node-role.kubernetes.io/<role>= form is preferred. kubernetes.io/role=node and node-role.kubernetes.io/node=
  are also present.

Workaround: create the configmap with kubectl create configmap -n kube-system kube-dns before updating.

Known Issues

Rolling updates

Rolling update to 1.6 does not succeed because new kube-dns pods mount a configmap with an optional volume map,
but that is enforced by the kubelets, which are upgraded after the master.

etcd3

kops is not yet recommending etcd3. Right now we are working on resolving issues such as HA upgrade support.

1.5.3

• Important for Terraform Users Make ELB naming unambiguous by including the full cluster name. This will cause the ELBs to be recreated if using Terraform with private topologies, causing disruption of external access to the API and of external access to the bastion (if enabled). Expected disruption is less than 5 minutes. Use export KOPS_FEATURE_FLAGS=+UseLegacyELBName to keep the legacy naming and avoid disruption. Fix #1899

• Fix terraform output of shared subnets. Fix #1977

• Add support for i3 instances (thanks @geojaz)

• Experimental drain rolling-update,

• Experimental GCE support

• Update Weave to v1.9.3

• Put flannel in guaranteed class (thanks @mihok)

• DNS autoscaler fixes (thanks @MrHohn)

• Remove legacy flags (thanks @mtaufen)

• Add route53 mapper addon (thanks @itskingori)

• Build fixes (thanks @zmerlynn)

• Disable cloudformation delete (thanks @kris-nova)

• Docs fixes (thanks @bowei, @jonchiu, @dosullivan, @DualSpark, @foxylion, @kris-nova

1.5.2-beta.2

Breaking changes:

• Route53 permissions are scoped only to the hosted zone in use, rather than all zones. You can add additionalPolicies
  to expand the permissions available.

Changelog:

• Scope route53 permissions to DNS_ZONE only (thanks @zytek)
• Improved kops edit when errors are made (thanks @geojaz, @sethpollack, @yissacharcw)
• Initial support for CoreOS (thanks @aledbf)
• Initial support for CloudFormation output
• Add support for adjusting ELB idle timeout for apiserver (thanks @zacblazic)
• Cloud labels can be defined at cluster level, and through CLI flags (thanks @robinpercy)
• More flag mapping - auditing & volume mapping (thanks @aledbf, @sethpollack)
• Fix issue with leaking elastic IPs on cluster delete (thanks @ikropotov)
• Fix terraform render for additional security groups (thanks @philk)
• Improve suggestions after cluster creation, and validation output (thanks @kamilhristov)
• Unset kubectl current-context as part of cluster delete (thanks @kamilhristov)
• Terraform: include output variables, for reuse in a module
• Allow general access from nodes to masters, but block sensitive ports (etcd)
• Bump dns-controller to 1.5.2: include namespace in keys
• Update alpha channel with k8s 1.4.9 and 1.5.3 (thanks @itskingori)
• Run Weave Net in Guaranteed Quality of Service (thanks @bboreham)
• Bump Weave Net to 1.9.0 (thanks @bboreham)
• Flannel support (thanks @mihok)
• Bump calico version to 2.0.2 (thanks @heschlie)
• Canal support (thanks @heschlie)
• Autoscaling improvements (thanks @andrewsykim)
• Increase conntrack table size using sysctl (thanks @aledbf)
• Update cgroup hierarchies for k8s 1.6 (thanks @derekwaynecarr)
• Improved validation (thanks @geojaz, @sethpollack, @sstarcher)
• Resolve DNS Hosted Zone ID while building IAM policy (thanks @zytek)
• Makefile fixes (thanks @aledbf, @apenney, @zytek)
• Choose only one subnet per AZ for API ELB
• Prevent spurious changes around IAM roles
• Delete CloudFormation stack on kops delete cluster
• More tolerant parsing of IAM policies
• Allow bastion image override when creating cluster
• Install ethtool via nodeup, so it is always installed
• Docs fixes - thanks @ahawkins, @andrewwatson, @chrislovecnm, @crigor, @fate-grand-order, @geojaz, @heartlock, @itskingori, @kris-nova, @mshenhera, @yissachar, @zacblazic, @zytek

1.5.1

• Recommend a k8s version based on each kops version
• Fix docker systemd manifest on RHEL for 1.12
• Populate sshAccess correctly when converting from older versions
• Fix PodManifestPath / Config schema (thanks @kamilhristov)
• More efficient CI (thanks @andrewsykim)
• Improved docs (thanks @apenney)
• Display proposed changes sorted by name (not topologically)

Known errata (will be fixed in 1.5.2):

• kops update cluster shows that additional (empty) IAM policies will be created (#1782)

1.5.0-beta2

• Add --master-count flag, to make it easy to run masters in the same AZ
• Support additional security groups (thanks @tsupertramp)
• Apply Terraform formatting to TF files (thanks @tazjin)
• S3 location discovery fix when bucket is not owned (thanks @tazjin)
• Allow S3 bucket object ACL override (thanks @tazjin)
• Use PodManifestPath instead of deprecated Config (thanks @mtaufen)
• Bumping Calico versions to 2.0.1 (thanks @heschlie)
• CLI: Add long descriptions (thanks @kamilhristov)
• Support p2 instances (thanks @anurag)
• Add Eviction flags to evict based on inodes
• Map NvidiaGPUs / --experimental-nvidia-gpus flags
• Install ebtables in nodeup
• Additional IAM policy deletion fixes
• Fix path to docker service manifest
• Fixes for kube-up import
• Version dependent recommendations in channels
• More tolerance for concurrent deletion
• Relax DNS requirements on shared VPCs
• Allow specification of multiple CIDRs to create cluster
• AssociatePublicIP is only explicitly set when specified, defaults based on topology
• Improved logging on AWS errors around retries
• Target autoscaler-addon to master nodes (thanks @Snaker)
• Better docs (thanks @apenney, @anurag, @geojaz, @jasondewitt, @scottmwebber, @shrugs, @tsupertramp, @yissachar)

1.5.0-beta1

• Fix bug with k8s 1.5.1, where we were incorrectly passing attach-detach-reconcile-sync-period, which was not defined until 1.5.2
• Tolerate running etcd members / masters in the same instance group
• Populate the API access fields, so they are discoverable
• Don't create route table at all if all subnets are shared
• kube-dns autoscaler: set min replicas to 2
• Fix bastion name in SSH suggestion
• Move validation & better error messaging

1.5.0-alpha4

• Shared NAT gateways - specify egress (thanks @geojaz)
• Extensible IAM permissions (thanks @yissachar)
• Confirmation before deleting instance groups (thanks @frodopwns)
• Map more flags for "expert mode" (thanks @blakebarnett)
• Update cluster-autoscaler (thanks @sjevs)
• Fix to apply the correct taint to master (broken in alpha3 - thanks @rchoh)
• Don't try to delete resources when target=terraform (thanks @tanner-bruce)
• Relax constraints on mixed-topology models
• Can now rolling-update a single instance group
• Multiple masters in the same AZ (by kops edit cluster, currently)
• Prefer to resolve nodes by IP, relaxing constraints on DhcpOptions
• Update default instance size for ca-central-1 and eu-west-1
• Automatically infer CIDR from VPC and Subnet IDs
• Cluster label now appears in kops get -oyaml output
• Optimize Route53 calls in dns-controller
• Docs fixes (thanks @caarlos0, @evildandelions, @foxylion, @vendrov)
• Code cleanup (thanks @fate-grand-order)