Fix ssh terminals

[parisa-mchp]

What it does

This PR addresses an issue where from a linux client connecting to a windows server with by ssh, you cannot open terminals. This is because the node-pty module that we rely on to create terminals is compiled in a platform specific way, so when the node backend is copied onto the server, an incorrect version of node-pty is provided. For some reason, this does not cause issues making the reverse connection, from a windows client to a linux server.

To fix this, we do a webpack exclude so that the node-pty module is resolved at run-time instead of at compile time, and as part of the server setup we install node-pty so a correct version will be available for the server's platform. We don't exclude node-pty when not connected to a server. Otherwise, additional care would need to be taken when creating a theia extension to ensure any generated appimages include the node-pty module, and that a remote, which would then have two copies of the module, resolve the correct one.

How to test

To reproduce the issue, connect from a linux client to a windows server, and attempt to open an ide terminal. Any other client-server combination should still work. I have tested linux-linux, linux-windows, windows-linux, and windows-windows, but have don't have access to any systems running macos. When connecting to a server twice with different builds, make sure to delete the .theia-example-electron-1.60.0-remote folder.

Follow-ups

The current implementation is somewhat complex. Suggestions would be appreciated, but at the moment I don't see a way to simplify it.

Breaking changes

• This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[msujew]

Thanks for looking into this!

However, I'm not sure this is a good solution to the issue. Is there something wrong with the binaries we're uploading? Can you try using the code from master and just replace the binaries for node-pty with ones that come with the installation of node-pty on Windows?

[msujew]

Question: I'm confused as to why this is necessary. Can you expand on that a bit? AFAIK our build shouldn't bundle the JS code differently on Windows compared to Linux. So even if the binaries are different, the JS code shouldn't change.

[msujew]

This might not work if the remote machine doesn't have a "normal" network connection. I.e. outbound HTTP traffic might be blocked to some degree.

[parisa-mchp]

Can you try using the code from master and just replace the binaries for node-pty with ones that come with the installation of node-pty on Windows?

This is a little bit tricky if you look at the way windowsPtyAgent gets compiled it looks like this:

var WindowsPtyAgent = /** @class */ (function () {
    function WindowsPtyAgent(file, args, env, cwd, cols, rows, debug, _useConpty, _useConptyDll, conptyInheritCursor) {
        var _this = this;
        if (_useConptyDll === void 0) { _useConptyDll = false; }
        if (conptyInheritCursor === void 0) { conptyInheritCursor = false; }
        this._useConpty = _useConpty;
        this._useConptyDll = _useConptyDll;
        this._pid = 0;
        this._innerPid = 0;
        if (this._useConpty === undefined || this._useConpty === true) {
            this._useConpty = this._getWindowsBuildNumber() >= 18309;
        }
        if (this._useConpty) {
            if (!conptyNative) {
                try {
                    conptyNative = __webpack_require__(Object(function webpackMissingModule() { var e = new Error("Cannot find module '../build/Release/conpty.node'"); e.code = 'MODULE_NOT_FOUND'; throw e; }()));
                }

That line at the bottom being the problematic one. If we add the conpty.node post hoc, it won't use it because at compile time the error is hard coded. To circumvent this, I tried manually copying conpty into the module between the npm install and npm run build:electron steps, but the above file was still compiled the same way by webpack. I wasn't sure why this is, I suspect it has either to do with the fact that the module is already built when we run npm install, or as a result of some sort of dead code elimination step that assumes the module isn't need because we're compiling on linux.

All of this is to say, I suspect what you've suggested is doable, but I'm not sure how to.

[msujew]

This is a little bit tricky if you look at the way windowsPtyAgent gets compiled it looks like this

Ah, thanks a lot, that explains it well! And thank you for the investigation efforts. Alright, I think that we probably can get a simpler solution by performing a bit of bundler magic in Webpack. I'll prepare something next week 👍

[parisa-mchp]

I think that we probably can get a simpler solution by performing a bit of bundler magic in Webpack.

Where's this at? Would you like me to look into implementing the idea you described?

[msujew]

@parisa-mchp I actually created a WIP over at master...msujew/fix-remote-windows-terminals but didn't have time to get back to it (I also currently don't have a setup in which I can easily test this). The change might work out of the box, but it shows the approach how I would fix this. If you have some time to dig into it, please go ahead and continue on my changes :)

[parisa-mchp]

@msujew Thanks so much for providing this. I don't know webpack too well so I would have had a hard time doing this part. From here I think I can get something working.

[msujew]

@parisa-mchp For some reason, the ECA validation doesn't work correctly with my email in the commit. Feel free to remove my attribution from the commit 👍

[parisa-mchp]

Okay, right now everything seems to be working on my end. As before, I don't have a way to test macos, but every combination of windows and linux I tried seemed okay.

As for the test failures, I think there's some kind of outage in one of the servers the plugins are downloaded from. It was working fine earlier today for me, but the download:plugins step doesn't seem to be working.

[msujew]

@parisa-mchp It looks like the Eclipse foundation service infrastructure faces a major outage right now. That's the reason for the failing pipelines. Nothing to worry about :)

[msujew]

Thank you! I'll merge this as soon as the Eclipse foundation outage is fixed (one of the checks is a required check unfortunately).

[JonasHelming]

@msujew Should be good to retrigger now!

[msujew]

@parisa-mchp Do you mind force pushing to the PR? The GH actions seem to be stuck.