PFS in inconsistent state when new RSB joins during runtime #1022
Description
Description
Due to the multi client setup a PFS can only receive and maintain state of a homerserver's users to which it is also connected to. If a new RSB joins during runtime, there is currently no way to setup a new client to the new RSB. In order to maintain state and communicate with these users, a PFS must restart.
PFS <-> Matrix
Here is a short overview how and for what the PFS uses Matrix.
- Partioning communication: PFS is connected to all homeservers
- Each PFS Matrix user receives capacity and fee updates from the homeservers' users through to device messages
- Each PFS Matrix user receives presence updates from the homerservers' users due to the presence router through the sync call
- A raiden node assembles a PFS matrix user id by fetching PFS addresses from the service registry combining it with its homeserver's server name.
Options
Polling for new RSBs
RSBs are added to the list of known servers in the rsb repo. Upon startup the PFS fetches this list. It could periodically fetch the list and see if it changed and act based on it.
Go back to single client strategy
The multi client strategy was introduced to bypass the original presence bug in matrix where presence updates got dropped in the federation. The presence bug should be fixed by now (NEEDS TO BE CONFIRMED) and this opens the opportunity to go back to a single client strategy.
The PFS would connect to only one homeserver and receive messages and presence updates from other servers via the federation (as matrix is intended to be used).
This change would remove the complexity out of the PFS.
Communication of Raiden Node with PFS via HTTP
Unfortunately, the current setup of sending to device messages to the PFS relies on the multi client strategy as described above. A Raiden node would need to find out the PFS's real user id instead of implicitly talking to the local user of the PFS. One option would be that the PFS provides the matrix user id in the info endpoint.
Another solution which was already discussed is to send capacity and fee updates directly to the PFS via an HTTP endpoint. This could bring additional advantages:
- Get an actual response if transmitting information was successful (instead of the send-and-forget approach with to device)
- Increasing privacy since the RSB provider cannot read capacity updates anymore
- Additionally, the dependency of Matrix in the PFS is reduced (not gone, we still need it for presence updates)
Proposal
IMHO I would vote for option 2, removing the multi client strategy and the complexity which comes with it. It was intended to be a temporary fix anyway. With the additional change to send updates to the PFS via HTTP we can increase reliability, performance and privacy while reducing the dependency on Matrix to an additional degree.
What needs to be done?
- Removing the multi client architecture in the PFS
- Changing the presence router to contain the list of actual PFS user ids
- PFS needs to provide matrix user id for presence router
- Change broadcasting in the raiden nodes to post updates via http