Investigate periodic pod eviction

[trent-s]

pods are occasionally ejected with an errors such as:

The node was low on resource: ephemeral-storage. Threshold quantity: 24068434671, available: 23201836Ki. Container kube-rbac-proxy was using 8Ki, request is 0, has larger consumption of ephemeral-storage. Container manager was using 106848Ki, request is 0, has larger consumption of ephemeral-storage.

and

The node was low on resource: ephemeral-storage. Threshold quantity: 32121498671, available: 30114200Ki. Container kube-rbac-proxy was using 4Ki, request is 0, has larger consumption of ephemeral-storage. Container manager was using 1920Ki, request is 0, has larger consumption of ephemeral-storage.

[trent-s]

But... This might be as simple as just reducing stdout logging. As we are in development, we are someone verbose at the moment...

[trent-s]

Also, possibly reduce sampling rate from 2 seconds to something a bit longer.... 30 seconds? 60 seconds?