tcsh: Fix entering invalid (non-UTF8) directories

[egmontkob]

Is there an existing issue for this?

• I have searched the existing issues

Midnight Commander version and build configuration

4.8.33, git

Operating system

any

Is this issue reproducible using the latest version of Midnight Commander?

• I confirm the issue is still reproducible with the latest version of Midnight Commander

How to reproduce

As pointed out at #4480 (comment) and #4804 (comment) (also the next comment), tcsh's strings aren't binary safe. It wants to ensure that all variables and whatnot are valid according to the current locale (which is almost certainly UTF-8).

This causes problems in two directions:

If, with mc's panels on, you navigate to a directory whose name isn't valid UTF-8, the tcsh subshell (as we inject a cd command to it) might not be able to enter it.

If, with mc's panels off, you navigate in the tcsh subshell to a directory whose name is invalid UTF-8 (using e.g. wildcard or tab completion), it might report back this directory (using the echo $cwd:q command) in a wrong encoding, so when mc's panels are turned on, it fails to switch to that directory.

Let's try to find a binary safe solution for both of these.

Expected behavior

.

Actual behavior

.

Additional context

No response

[egmontkob]

For the first direction, I have a brand new idea which seems to work:

Place the cd command in a temporary file. For high bytes, use their raw bytes (not backslash-octal notation). Inject a source [filename] command to tcsh.

It seems to me that when reading commands from a file, rather than interactively, it retains binary safety in parameters.

Also, we avoid the need for breaking lines after every 250-ish bytes to please the short cooked mode buffer of some kernels.

---

For the second direction, it seems to me that the external pwd command is fine, even including symlinks (i.e. tcsh sets the environment variable PWD in a binary safe way).

---

Let's double check both of these claims, and if they're true then switch to this method.

It has been pointed out at #4634 (comment) and followup comments that frequently creating temporary files is not the best approach for disk usage. We could resort to the current method of injecting a cd command if the path is ASCII-only and the entire command would remain below 250-ish bytes. Alternatively we could check if source'ing works over a named pipe. Or just ignore this problem altogether for such a marginal (IMHO) and terrible (not just my opinion) shell.

[zyv]

Alternatively we could check if source'ing works over a named pipe.

Sounds like an interesting approach :)

[egmontkob]

I can very easily imagine that (in some shells at least) source requires a regular file (being able to query its size upfront; allocate a buffer of that size to read that into, or mmap it). Dunno, haven't tried.

[krobelus]

It seems to me that when reading commands from a file, rather than
interactively, it retains binary safety in parameters.

If we add a tcsh-specific workaround, it shouldn't affect other shells that can store arbitrary bytes in shell variables, and execute "cd" commands with non-UTF-8 paths.
(of course if mkfifo fifo && . fifo works everywhere and is more elegant, by all means)

followup comments that frequently creating temporary files is not the best
approach for disk usage.

yeah but if that's really relevant, we can do it only if path is non-ASCII, or even not-valid-in-current-locale.
Anyway, I agree it's probably not worth spending much time on this,
unless we expect tcsh improve in future (if so we should ask them to provide binary-safety).

Or just ignore this problem altogether for such a marginal (IMHO) and
terrible (not just my opinion)
shell.

I actually met someone who uses it, and part of their rationale was "bash is a terrible interactive shell". 🤔
My only experience is that tcsh takes 3 seconds to start up because it makes some network requests..