Can mergerfs create a new folder on another drive, if drive is full

[FlyveHest]

While i'm pretty sure I know the answer, I thought i'd go ahead and ask anyway :)

If I have moveonenospc enabled, and a new file causes the disk its writing to to become full, and the particular path the file is being written to only exists on that one disk, can mergerfs create it on another?

Theres no policy that matches this behaviour, and I would think that the overall thought behind mergerfs being on top of the filesystem will prohibit this, but it is somewhat annoying when a particular disk is getting full, that you have to either manually create the path on another, or manually move the entire folder from the full disk to a new one.

[trapexit]

There is no policy that matches that because path creation is not what policies control.

A policy controls selection of branches. The creation of a directory on a branch happens after selection if needed.

There is no reason for you to be creating paths unless you use ep policies and very few people have any reason to use an ep policy.

https://trapexit.github.io/mergerfs/faq/why_isnt_it_working/#why-are-all-my-files-ending-up-on-1-filesystem

I suggest rereading the sections on policies and such

https://trapexit.github.io/mergerfs/config/functions_categories_and_policies/

moveonenospace is for after a file is already open and being written to and after minfreespace already was used to decide the placement of the create.

https://trapexit.github.io/mergerfs/faq/configuration_and_policies/#what-settings-should-i-use

[FlyveHest]

Before anything, you mention that only few people need an ep policy, but the document you linked specifically say that the default create policy is epmfs? (Mostly just curious here)

I've re-read the sections again, and I think I probably didn't explain my scenario correctly.

I have three disks, with the following structure.

• Disk A
  • Folder A
    • Subfolder A
• Disk B
  • Folder A
    • Subfolder B
• Disk C
  • Folder A
    • Subfolder C

I am now writing a couple of large files to Folder A/Subfolder A, and in the process, Disk A becomes full, what happens now is that the file writing process just fails, and unless I create a Subfolder A in one of the other Folder A, I can't continue.

The process that writes never implicitly create a Subfolder A, it just writes to it, as it was already existing.

I have my create policy as eplfs and moveonenospc=true, rest is default.

[trapexit]

https://trapexit.github.io/mergerfs/faq/why_isnt_it_working/#why-are-all-my-files-ending-up-on-1-filesystem

As I explain here the default is epmfs because it doesn't change the layout of the system. People should make an informed choice as to what they want. epmfs means nothing will change. It is easier to stop strict behavior than fix loose behavior.

https://trapexit.github.io/mergerfs/faq/technical_behavior_and_limitations/#why-do-i-get-an-out-of-space-no-space-left-on-device-enospc-error-even-though-there-appears-to-be-lots-of-space-available

moveonenospc is the feature that handles that situation. I'm not sure what your question really is.

https://trapexit.github.io/mergerfs/config/options/#mount-options

moveonenospc=BOOL|POLICY: When enabled if a write fails with ENOSPC (no space left on device) or EDQUOT (disk quota exceeded) the policy selected will run to find a new location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs)

[FlyveHest]

To re-iterate, I am writing File X to Subfolder A in my thought-experiment-setup above, and Disk A runs out of space.

Shouldn't setting moveonenospc to true, which is the mfs policy, result in the creation of a new Subfolder A on Disk B, and File X being written there instead? (Assuming Disk B is the disk with the most free space)

Or, does mergerfs report "disk full" back to the program trying to write before it starts writing, which in turn aborts?

[trapexit]

moveonenospc=BOOL|POLICY: When enabled if a write fails with ENOSPC (no space left on device) or EDQUOT (disk quota exceeded) the policy selected will run to find a new location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs)

Yes. As it says: "if a write fails with ENOSPC..." that feature kicks in. Again, policies don't have anything to do with creation of directories. A location is selected based on the policy and if needed the directory path is cloned.

If any other random call were to return ENOSPC it is not impacted by moveonenospc.

[trapexit]

It would help if you explained what the confusion was relative to the documentation. As I say in the docs they are pretty thorough and literal. Don't read between the lines. It behaves as described. Nothing more or less.

[FlyveHest]

I'll try and explain the scenario, it very well might just be me that doesn't understand precisely how mergerfs works.

There is a directory on a disk, it only exists on that disk, I want to write a file on that disk but can't because its full.

I can't write any more files to that specific directory before I manually create the same directory structure on one of the other disks.

At least, thats how it seems to work on my system.

If this is intended, then I've clearly misunderstood how writing to mergerfs works and I apologize, its still a great piece of software.

[trapexit]

Everything you ask is in the docs I've shared numerous times.

"Writing to a file" first requires the opening of a file or the creation of a file. And that open or create is managed by the policy you choose. And those policies work exactly as described in the docs. If you are manually creating dirs you clearly have set the policy to some "ep*" policy which as described in the docs is not recommended. Please can you just follow the docs and use the recommended settings as described in the quick start and if you want any help provide details as requested on the support page. I can't help if you don't explain to me precisely what you have setup and what you don't understand about the documentation. The docs are very thorough and honed over a decade. Almost everything anyone has ever asked and every feature is explained there.

https://trapexit.github.io/mergerfs/support/

[trapexit]

There is a directory on a disk, it only exists on that disk, I want to write a file on that disk but can't because its full.

Because of how you've decided to configure the product.

I can't write any more files to that specific directory before I manually create the same directory structure on one of the other disks.

Because you've chosen a path preserving policy which as explained in the docs requires manual intervention because it preserves paths.

At least, thats how it seems to work on my system.

Because of how you've configured it.

If this is intended

Intended in so far as that's how you've configured it to behave.

[FlyveHest]

So, if i'm understanding correctly, moveonenospc is not triggered by the minfreespace setting?

It is only triggered if the disk runs out of space after writing has begun, which I would think is only possible if another process is writing to the disk at the same time?

If that is the case, then it explains my situation, and I have assumed something that was not true, which I of course regret.

But I don't think that is communicated all that clearly in the docs, at least, not for someone who does not have intricate knowledge on Linux filesystems and what actually happens when a new file is being written

That a write operation is not a single command, but actually a chain of creating a new file and THEN writing was not something I was consciously aware of.

[trapexit]

As it says in the docs.... anything NOT "ep*" or "msp*". Why don't you just use the recommended settings like I recommend in the docs and save us all this back and forth.

[FlyveHest]

I'm going to close this now, and thank you for all the input (and sorry for potentially having wasted your time), but I would just like to point out that using the recommended settings would not work, or, at least the docs say that the default create policy is epmfs

[trapexit]

What do you mean "would not work". They certainly do work. Yes, the default policy is epmfs. What does that have to do with the recommended settings?

[FlyveHest]

You're suggesting that I use the recommended settings to solve the issue I have, I just pointed out that the default settings, which I would assume are the recommended ones, are ep based,.

If the default settings are not the recommended ones, then I don't really know what to say.

[trapexit]

The docs very explicitly lay out what and why. The second page of the docs is the quickstart guide which tells you what to use and why for the average person with excessive examples. There is literally a FAQ entry why default is epmfs and why it was chosen to be the default. If you are unable or unwilling to read and understand those then I don't really know what to say.