TightVNC File Upload Protocol

[MikeRich88]

Is the TightVNC File Upload Protocol documented anywhere? How did you get it implemented?

I want to add this protocol to my VNC Client. (I am writing one in Xojo just for fun). My VNC client supports Mac OS first and Windows as well. I actually want to add that 1-bit TRLE mode to it, so it could work more efficiently with your server, but that also seems poorly documented.

Also, the solution to your issue of resource forks is simply to have the client upload as a MacBinary file and then the server could just leave it alone, or it could unpack the MacBinary file for convenience. I could implement this in my client.

My client (currently named RBVNC for lack of any better name) isn't open source because I may want to sell the actual VNCCanvas code as a component to other Xojo developers, but I would have no problem releasing a special free build of it with the file and TRLE 1 bit support that people could use with your server.

[marciot]

Yeah, the documentation situation for VNC is fairly poor, which I suspect is the reason compatibility among clients and servers is so poor.

It would be amazing to have a VNC client on that Mac that supported all the features of MiniVNC. My current idea was to try adding a web interface to MiniVNC so I could have a compatible Javascript client, but that would be a huge undertaking, requiring me to write a web server into MiniVNC, and then implement the WebSockets protocol on that server. Having a working standalone client would be far better, so I am glad to help your project in any way I can!

I ended up relying on several resources, many of which are incomplete. Here is quick guide. The TRLE protocol is actually well documented, if you look in the right place:

https://datatracker.ietf.org/doc/html/rfc6143#page-27

On a B&W Macs, I only use TTRE tile sub-tile types 1, 2 and 127 and all frame update rectangles are aligned to a 16-pixel boundary (this is to avoid bit-shifting and bit padding which would be very slow on a 68000). On color Macs, I use all the tile types and relax the alignment.

The TightVNC file transfer is a bit harder to get one's head around. TightVNC extends the basic protocol in three ways:

1. TightVNC security type
2. TightVNC encoding
3. TightVNC file transfer

Number 1 is required for 2 and 3, so I had to implement it in MiniVNC (the encoding is not needed, but I might consider it in the future, as it does have a 1-bit mode that might improve compatibility on old Macs).

The TightVNC security type is fairly well documented here:

https://vncdotool.readthedocs.io/en/0.8.0/rfbproto.html#tight-security-type

I implement the TightVNC security type, but then use VNC authentication for the password. My server does not validate the challenge response, so any password works.

You also will need to parse the TightVNC capabilities data during server init. This stage is well documented here:

https://vncdotool.readthedocs.io/en/0.8.0/rfbproto.html#serverinit

The general description in that data format is correct, but the capability codes for file transfer operations are incorrect (or maybe an older version). I have been unsuccessful in finding any client that actually uses those messages. I had to rely on the TightVNC source code for the correct capability codes used in TightVNC 2.0. They are the uncommented ones here:

https://github.com/marciot/mac-minivnc/blob/v1.4-beta-mar-28-2024/mac-cpp-source/TightVNCTypes.c

For the actual way in which messages are sent, I was helped greatly by this discussion on StackOverflow:

https://stackoverflow.com/questions/70720527/specification-of-client-uploads-in-tightvnc-rfb

The original VNC specification uses 1 byte message codes, but TightVNC extends this to a four-byte code. The StackOverflow discussion explains how this is done. From that point on, the last piece of the puzzle is the format of the messages themselves. Again, I used the TightVNC source code for this, but at this point you are probably better off just looking at my implementation:

https://raw.githubusercontent.com/marciot/mac-minivnc/v1.4-beta-mar-28-2024/mac-cpp-source/TightVNCSupport.cpp

Just look for "IN_STREAM_COPY" and "OUT_STREAM_COPY" and it will basically tell you the message format.

Hope this helps!

[MikeRich88]

I think the first problem with my client is that it disregards the PixelFormat the client sends, and always demands it's own that is easier to parse. (I've seen servers send really stupid stuff like BGR instead of RGB, or even GBR, or other really odd arrangements. I use some tricks in Xojo and I can deal with RGB data efficiently, everything else would require byte swapping.)

Do you use an emulator setup or are you always connecting to a real 68K Mac? I have some Macs, but the RBVNC client is developed on a modern system, so if I can get one of the emulators to network with my client then that would make it far easier to start to work on these changes.

I'll set up a little website with some builds of RBVNC on it. These won't have files or 1-bit, but should work with the color stuff at least. It does support several encodings, off the top of my head: Raw, RRE, CoRRE, HexTile, ZRLE, Tight PNG (but not Tight). Tight PNG only seems to be used by QEMU. I had an idea once of remotely hosting Mac OS 9 on a server running QEMU that you could access with VNC. Sound could even be transmitted, because QEMU has it's own sound extension to VNC.

[marciot]

Yeah, most clients send a PixelFormat right off the bat. Unfortunately, the VNC protocol does assume the server will do most of the heavy lifting. Unfortunately, MiniVNC is meant to run on machines that aren't very powerful (hundreds of times less powerful than the machines where I am running the clients), so it's exactly backwards as to what it should be!

I use Basillisk II for my development. This is needed since I use a Windows PC for development. I will typically run MiniVNC inside Basillisk, then I will run VNC clients locally on the Windows desktop and connect it to the MiniVNC instance running in Basillisk. This seems to work fairly well. Once I get the major kinks worked out, I might run MiniVNC on my older MacBook, my PowerBook or my Mac Plus. But generally, I only do that much later for final testing as working with real hardware is always more of a hassle than working with the emulator.

[MikeRich88]

I haven't forgotten about this. My client (RBVNC) worked with your server (in the color modes) right away, with the server running in QEMU, on Mac OS 9.x (forgot the exact version). So that's a good extra test for your server as well running under the PPC emulator, etc.

I did find a crash in your server but I forget now what it was exactly.

I should probably not look at your source code though since it's GPL and mine isn't, and that would cause some issues. I still haven't decided what to do with it though.

I think the file transfer could be awesome. There's literally no clients on macOS that support that. Especially if on your end it could detect a MacBinary file and unpack it appropriately. Modern macOS still fully supports and even still uses resource forks in some limited cases. It should be possible to for example, unpack a .sit file on a much faster modern Mac and then transmit an application with resource fork over VNC.

No changes to the protocol at all would need to occur. My client would detect the file has a resource fork, package into a MacBinary file, and then transmit as "nameOffFile.bin". The server end should look for both a file name ending in ".bin", and the magic string, which for MacBinary III format is the string "mBIN" at offset 104 bytes. Only one version could be supported (MacBinary III probably) for simplicity. From my client, non-miniVNC servers would also receive the MacBinary file, which is an improvement over receiving some damaged and useless file.

There is also the very similar but far less otherwise supported AppleSingle format, but I'd think its far easier to find suitable example code for MacBinary.