Using S3 storage: Files remaining in uploads folder?

[windsource]

I use OpenCloud with S3 storage. When syncing (uploading) a folder using the desktop app, I saw files being created in /var/lib/opencloud/storage/users/uploads which are deleted after a short time. So I assume files are stored there temporarily before being stored on S3.
Now after the the sync has been finished I recognized some files remaining in that folder:

$ ls -l
total 15244
-rw-r--r--    1 openclou openclou   3162112 Jun 20 07:20 3bbf55c0-a354-4b80-bda2-b57c056e4286
-rw-------    1 openclou openclou      1068 Jun 20 07:20 3bbf55c0-a354-4b80-bda2-b57c056e4286.info
-rw-r--r--    1 openclou openclou   7798784 Jun 20 07:20 8c7a8062-f8b7-41c2-bfd8-838ec0a14c60
-rw-------    1 openclou openclou      1068 Jun 20 07:20 8c7a8062-f8b7-41c2-bfd8-838ec0a14c60.info
-rw-r--r--    1 openclou openclou   4636672 Jun 20 07:20 b4f2f7bd-cc26-409e-a27d-f6cb4d25cf11
-rw-------    1 openclou openclou      1069 Jun 20 07:20 b4f2f7bd-cc26-409e-a27d-f6cb4d25cf11.info

When I read the .info files and the filenames listed in there, I can see those files being synced, so they exist in the web UI.

Does anyone know, why the files keep being in the uploads folder even a day after the sync?

[micbar]

Something must be wrong with your S3 credentials. Do you use clamav?

[windsource]

I don't use clamav. I use Hetzner object storage and I can I see files being created there. The sync seems to be OK. I have synced more than 6000 files and only 3 of them remain in the uploads folder. I can even download the 6000 files so I think the credentials are OK.

My setup looks like:

STORAGE_USERS_DRIVER=decomposeds3
STORAGE_SYSTEM_DRIVER=decomposed
STORAGE_USERS_DECOMPOSEDS3_CREATE_BUCKET=false
STORAGE_USERS_DECOMPOSEDS3_ENDPOINT=https://hel1.your-objectstorage.com
STORAGE_USERS_DECOMPOSEDS3_REGION=hel1
STORAGE_USERS_DECOMPOSEDS3_BUCKET=my-bucket-name
IDM_ADMIN_PASSWORD=my-admin-password
STORAGE_USERS_DECOMPOSEDS3_ACCESS_KEY=my-accesskey
STORAGE_USERS_DECOMPOSEDS3_SECRET_KEY=my-secretkey

OpenCloud 3.0.0
OpenCloud Desktop 2.0.0-rc.1 4b6384

[micbar]

Ah, Hetzner S3. I saw that also. It is somehow slow and we implemented a self healing restart of the postprocessing.

[micbar]

The docs are quite verbose, but you can create a job to restart uploads into S3.

[micbar]

https://docs.opencloud.eu/docs/dev/server/Services/storage-users/Storage-users-info

[windsource]

Thanks @micbar for pointing me to the documentation! Actually I have not been aware of that section. Then

opencloud storage-users uploads sessions --restart

solved my problem and now the upload folder is empty.

Support of S3 for storing files was a key feature for me when choosing OpenCloud. Before using Hetzner object storage I have been using Backblaze B2 object storage. I have compare the performance of both and they are pretty similar. Both achieve about 100 MBit/s in both upload download. Actually I recognized the same problem with remaining files in the uploads folder when using Backblaze B2.

So now I will add a cron job to my K8S setup to perform that operation on a regular base. Would it make sense to integrate that in OpenCloud directly somehow?

[micbar]

So now I will add a cron job to my K8S setup to perform that operation on a regular base. Would it make sense to integrate that in OpenCloud directly somehow?

should become part of the helm chart @butonic

[mitexleo]

Offtopic:

Is the configuration same for backblaze B2? I'm not sure if my config is working or not.

I'm uploaded a small file 10 minutes ago and it's still processing.

[windsource]

For Backblaze B2 (region Europe) I changed:

STORAGE_USERS_DECOMPOSEDS3_ENDPOINT="https://s3.eu-central-003.backblazeb2.com"
STORAGE_USERS_DECOMPOSEDS3_REGION="eu-central"

[mitexleo]

Usually, how much time does it require to upload the files to s3?

[windsource]

Usually files are stored there right away.

[mitexleo]

Are you using any external proxy like nginx? I failed to deploy it using external reverse proxy.

[windsource]

I use traefik and for OpenCloud:

OC_INSECURE: "false"
OC_URL: 'https://opencloud.example.com'
PROXY_TLS: "false"
PROXY_HTTP_ADDR: "0.0.0.0:9200"