Discrepancy between self-hosting.md and blog post

[nisbet-hubbard]

Steps to reproduce

We were trying to self-host MatrixRTC without also running Element Call, but the discrepancy between self-hosting.md and Will Lewis’s blog post proved confusing.

The following part from his Apache configuration doesn’t have a counterpart in the nginx version:

    <Location "/sfu/get">
        Header set Access-Control-Allow-Origin "*"
        Header set Access-Control-Allow-Methods "POST"
        Header set Access-Control-Allow-Headers "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token"

        ProxyPreserveHost on
        ProxyAddHeaders on
        ProxyPass "http://localhost:8070/sfu/get"
        ProxyPassReverse "http://localhost:8070/sfu/get"
    </Location>

We tried:

location = /sfu/get {
        add_header Access-Control-Allow-Origin *;
        add_header Access-Control-Allow-Methods POST;
        add_header Access-Control-Allow-Headers "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_pass http://localhost:8070/sfu/get;
    }

Outcome

What did you expect?

A working RTC instance like https://livekit-jwt.call.matrix.org, which we hope won’t be discontinued just yet.

What happened instead?

With the addition mentioned above, we got

"OPTIONS /sfu/get HTTP/1.1" 200 490->0 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 18_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148" - https://mrtc.domain:443 "/usr/local/nginx/html/sfu/get"

Without the addition (ie following self-hosting.md alone), this request made by Element X client returned a 404.

But neither got us a working setup and we got stuck in ‘Waiting for media’ in every call.

The audio and video now appear, but only when one gets /sfu/get proxied. (Earlier we had configuration issues with livekit and lk-jwt, which were fixed.)

Operating system

Almalinux 10

Will you send logs?

Will send on request.

[spaetz]

Hi, I am not involved in element-call, but the selfhosting.md does contain an explicit example for nginx:

    location ^~ /livekit/sfu/ {
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      proxy_send_timeout 120;
      proxy_read_timeout 120;
      proxy_buffering off;

      proxy_set_header Accept-Encoding gzip;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";

      # LiveKit SFU websocket connection running at port 7880
      proxy_pass http://localhost:7880/;
    }

[spaetz]

What is important that your call to the SFU allows upgrading to a websocket connection, so this is important:

      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";

If I may insert a shameless self-plug: my testmatrix tool helps you identify some common issues and grab a jwt token that you can then use at the livekit connection tester.

[nisbet-hubbard]

the selfhosting.md does contain an explicit example for nginx:

    location ^~ /livekit/sfu/ {
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      …

      # LiveKit SFU websocket connection running at port 7880
      proxy_pass http://localhost:7880/;
    }

Yeah, I agree. The part I don’t understand is that example didn’t actually proxy /sfu/get, but Element X does make OPTIONS and POST requests to this particular endpoint (and not /livekit/sfu/get). So following that example invariably leads to 404 and no audio/video for us.

Shouldn’t we also proxy /sfu/get like the blog post did?

[spaetz]

The self-hosting examples uses LIVEKIT_URL=wss://matrix-rtc.example.com/livekit/sfu so that the SFU endpoint becomes matrix-rtc.example.com/livekit/sfu/sfu/get. Had they only configured LIVEKIT_URL=wss://matrix-rtc.example.com the endpoint would have been matrix-rtc.example.com/sfu/get.

Both is equally fine, you just need to make them consistent....