Running sftpgo as deployment in GKE - Best Practices

[rashmimaharana]

Hi @drakkan and Team,

I have a couple of questions about running sftpgo as deployment in GKE. I have visited the old discussion and got some ideas but still not clear about below.
I am using GCS bucket as storage and cloudSQL postgres as DB. Running 3 sftpgo pods using deployment files in GKE and accessing them using K8s service of type LoadBalancer.

Questions:
1> Apart from DB, does sftpgo is saving any data on container. If yes then we might have to use persistent volume(PV) to mount the same across multiple replicas of pod and can be used for disaster recovery.

2> What all folders or directories that we need to share or mount across multiple pods. I checked in your docs where you mentioned to mount "/var/lib/sftpgo" and "/srv/sftpgo" inside the containers.

3> Had an issue while running sftpgo as deployment(3 pod).It was not able to create home directory for sftp user. Then I ran the deployment as "root" privilege and it worked fine. But there is one issue. I could see its creating home directory for sftp user (user1) in path "/home/user1" inside one container or pod which ever got the request, not in others as i am running 3 pods using deployment backed by service. Do I have to mount "/home/" directory as shared volume across multiple pods so that all containers will be in sync.

Sorry incase if I am creating any confusion.

Below is the config changes that I am passing as environment variable using configMap.

configmap.yaml

apiVersion: "v1"
kind: "ConfigMap"
metadata:
name: "sftpgo-server-config-map"
namespace: "default"
labels:
app: "sftpgo-server"
data:
SFTPGO_DATA_PROVIDER__DRIVER: "postgresql"
SFTPGO_DATA_PROVIDER__NAME: "sftpgo_db"
SFTPGO_DATA_PROVIDER__HOST: "10.XXX.X.XXX"
SFTPGO_DATA_PROVIDER__PORT: "5432"
SFTPGO_DATA_PROVIDER__USERNAME: "sftpgo"
SFTPGO_DATA_PROVIDER__PASSWORD: "sftpgo"
SFTPGO_DATA_PROVIDER__PREFER_DATABASE_CREDENTIALS: "true"
SFTPGO_DATA_PROVIDER__CREATE_DEFAULT_ADMIN: "true"
SFTPGO_DEFAULT_ADMIN_USERNAME: "admin"
SFTPGO_DEFAULT_ADMIN_PASSWORD: "XXXXX"
SFTPGO_HTTPD__SIGNING_PASSPHRASE: "XXXXX"
SFTPGO_SFTPD__BINDINGS__0__PORT: "2022"
SFTPGO_HTTPD__BINDINGS__0__PORT: "8080"
SFTPGO_TELEMETRY__BIND_PORT: "10000"

Was getting below error while trying to upload files using sftpgo UI. But got fixed after running the deployment as root and once home directory created for sftp user "user1" inside all containers

Logs:
kubectl logs service/sftpgo-app-service | grep -i error
Found 3 pods, using pod/sftpgo-server-5db455595d-bqght

{"level":"debug","time":"2021-11-08T11:40:10.606","sender":"HTTP","connection_id":"HTTP_c64gp6ghf17kog3gqir0","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"}
{"level":"warn","time":"2021-11-08T11:40:19.755","sender":"HTTP","connection_id":"HTTP_c64gp8ohf17kog3gqirg","message":"error opening existing file, source: "transfer/test3.txt", err: open /home/user1/pipefile4174480699: no such file or directory"}
{"level":"debug","time":"2021-11-08T11:40:19.755","sender":"HTTP","connection_id":"HTTP_c64gp8ohf17kog3gqirg","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"}
{"level":"debug","time":"2021-11-08T11:40:50.027","sender":"HTTP","connection_id":"HTTP_c64gpgghf17kog3gqis0","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"}

[drakkan]

Hi @drakkan and Team,

I have a couple of questions about running sftpgo as deployment in GKE. I have visited the old discussion and got some ideas but still not clear about below. I am using GCS bucket as storage and cloudSQL postgres as DB. Running 3 sftpgo pods using deployment files in GKE and accessing them using K8s service of type LoadBalancer.

Questions: 1> Apart from DB, does sftpgo is saving any data on container. If yes then we might have to use persistent volume(PV) to mount the same across multiple replicas of pod and can be used for disaster recovery.

since you set SFTPGO_DATA_PROVIDER__PREFER_DATABASE_CREDENTIALS, only temporary files are saved locally, you need a volume only if you upload very big files (bigger than the available local space)

2> What all folders or directories that we need to share or mount across multiple pods. I checked in your docs where you mentioned to mount "/var/lib/sftpgo" and "/srv/sftpgo" inside the containers.

/var/lib/sftpgo contains the SQLite or bolt db, you are using postgres. /srv/sftpgo contains, by default, the user's home dirs, you are not using local users so only temporary files are written there, you need a volume only if you upload very big files

3> Had an issue while running sftpgo as deployment(3 pod).It was not able to create home directory for sftp user. Then I ran the deployment as "root" privilege and it worked fine. But there is one issue. I could see its creating home directory for sftp user (user1) in path "/home/user1" inside one container or pod which ever got the request, not in others as i am running 3 pods using deployment backed by service. Do I have to mount "/home/" directory as shared volume across multiple pods so that all containers will be in sync.

Sorry incase if I am creating any confusion.

Below is the config changes that I am passing as environment variable using configMap.

configmap.yaml

apiVersion: "v1" kind: "ConfigMap" metadata: name: "sftpgo-server-config-map" namespace: "default" labels: app: "sftpgo-server" data: SFTPGO_DATA_PROVIDER__DRIVER: "postgresql" SFTPGO_DATA_PROVIDER__NAME: "sftpgo_db" SFTPGO_DATA_PROVIDER__HOST: "10.XXX.X.XXX" SFTPGO_DATA_PROVIDER__PORT: "5432" SFTPGO_DATA_PROVIDER__USERNAME: "sftpgo" SFTPGO_DATA_PROVIDER__PASSWORD: "sftpgo" SFTPGO_DATA_PROVIDER__PREFER_DATABASE_CREDENTIALS: "true" SFTPGO_DATA_PROVIDER__CREATE_DEFAULT_ADMIN: "true" SFTPGO_DEFAULT_ADMIN_USERNAME: "admin" SFTPGO_DEFAULT_ADMIN_PASSWORD: "XXXXX" SFTPGO_HTTPD__SIGNING_PASSPHRASE: "XXXXX" SFTPGO_SFTPD__BINDINGS__0__PORT: "2022" SFTPGO_HTTPD__BINDINGS__0__PORT: "8080" SFTPGO_TELEMETRY__BIND_PORT: "10000"

Was getting below error while trying to upload files using sftpgo UI. But got fixed after running the deployment as root and once home directory created for sftp user "user1" inside all containers

you set the home dir to /home/user1 but sftpgo runs by default using 1000 as uid/gid so this user cannot write in /home/user1, please leave the home dir empty it will be automatically set to /srv/sftpgo/data/<username> (see here) and sftpgo can write to that location. You can set the home dir where you want but you have to ensure that the sftpgo user can write there (chown 1000:1000 <path to your custom home dir>)

Logs: kubectl logs service/sftpgo-app-service | grep -i error Found 3 pods, using pod/sftpgo-server-5db455595d-bqght

{"level":"debug","time":"2021-11-08T11:40:10.606","sender":"HTTP","connection_id":"HTTP_c64gp6ghf17kog3gqir0","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"} {"level":"warn","time":"2021-11-08T11:40:19.755","sender":"HTTP","connection_id":"HTTP_c64gp8ohf17kog3gqirg","message":"error opening existing file, source: "transfer/test3.txt", err: open /home/user1/pipefile4174480699: no such file or directory"} {"level":"debug","time":"2021-11-08T11:40:19.755","sender":"HTTP","connection_id":"HTTP_c64gp8ohf17kog3gqirg","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"} {"level":"debug","time":"2021-11-08T11:40:50.027","sender":"HTTP","connection_id":"HTTP_c64gpgghf17kog3gqis0","message":"connection removed, local address "10.X.X.X:8080", remote address "10.X.X.X:58049" close fs error: , num open connections: 0"}