Yjs integration with Milkdown Collab plugin

[lexhuismans-wk]

I'm working on a collaborative editor with Milkdown but I am encountering some issues with the editors not syncing properly at times.

It looks like at some point the server and client have a different version of the ydoc and therefore cannot integrate new updates coming in.

Client

• Client stores the Y.encodeStateAsUpdate(doc) in a database as well as having a default template
• When setting up the collab editor in the client
  • Check if we have an existing Y.encodeStateAsUpdate stored, if so apply this. I believe this should contain history etc.
  • If a template is provided apply the template

Client setup

// Setup collaboration
  const collabServiceRef = useRef<CollabService>();
  const editor = loading ? null : get() || null;

  useEffect(() => {
    if (!editor || !wsUrl) return undefined;

    const doc = new Y.Doc();
    yDocRef.current = doc;

    let wsProvider: WebsocketProvider;
    let retryDelay = 1000;
    const maxDelay = 30000;

    const connect = () => {
      wsProvider = new WebsocketProvider(wsUrl, props.docId, doc, { connect: true });

      editor.action((ctx: Ctx) => {
        collabServiceRef.current = ctx.get(collabServiceCtx);
        collabServiceRef.current?.bindDoc(doc).setAwareness(wsProvider!.awareness);

        if (props.base64yDoc) {
          console.log('Applying base64yDoc to editor');
          Y.applyUpdate(doc, base64docToUint8Array(props.base64yDoc));
        }
      });

      wsProvider.on('status', (payload: { status: string }) => {
        console.log(`Editor Status: ${payload.status}`);
        props.setColabStatus(payload.status);
      });

      wsProvider.awareness.setLocalStateField('user', {
        name: displayName ?? userEntityRef,
        color: `#${randomColor()}`,
      });

      wsProvider.once('sync', isSynced => {
        if (isSynced) {
          console.log('Synced with Yjs server');
          if (props.template) {
            console.log('Applying template to editor');
            collabServiceRef.current?.applyTemplate(props.template);
          }
          collabServiceRef.current?.connect();
        }
      });

      wsProvider.on('connection-error', (ev) => {
        console.warn(`WS connection failed: ${ev}, retrying...`);
        wsProvider?.disconnect();
        retryDelay = Math.min(retryDelay * 2, maxDelay);
        setTimeout(connect, retryDelay);
      });
    };

    connect();

    return () => {
      collabServiceRef.current?.disconnect();
      wsProvider?.disconnect();
    };
  }, [editor, wsUrl, props.docId]);

Server

• On the server it will do the following upon receiving a connection from a new client
  • Check if the document already exists in memory
  • If not check if it can be fetched from a Redis list

Yjs server setup

setupWsConnection = async (
    conn: WebSocket,
    req: http.IncomingMessage,
  ): Promise<void> => {
    conn.binaryType = 'arraybuffer';
    const docId = req.url?.slice(1).split('?')[0] as string

    // Get doc, initialize if it does not exist yet
    const [doc, isNewDoc] = this.getYDoc(docId);
    doc.conns.set(conn, new Set());

    if (isNewDoc) {
      this.logger.info(`Document doesn't exist in memory yet, fetching latest updates from Redis ${docId}`);
      const redisUpdates = await this.redisClient.getDocUpdatesFromQueue(doc);
      const redisYDoc = new Y.Doc();
      redisYDoc.transact(() => {
        for (const u of redisUpdates) {
          const encoder = encoding.createEncoder();
          encoding.writeVarUint(encoder, messageSync);
          syncProtocol.readSyncMessage(decoding.createDecoder(u), encoder, redisYDoc, this.redisClient.getRedis());
        }
      });

      this.logger.info(`Applying ${redisUpdates.length} updates from Redis to document ${docId}`);
      this.logger.debug(`redisYDoc ${redisYDoc.getXmlElement('prosemirror')} and doc ${doc.getXmlElement('prosemirror')}`);
      Y.applyUpdate(doc, Y.encodeStateAsUpdate(redisYDoc));
    }

    // Listen and reply to events
    conn.on('message', (message: WSData) => {
      this.messageListener(conn, doc, new Uint8Array(message as ArrayBuffer));
    });

    // Check if connection is still alive
    let pongReceived = true;
    const pingInterval = setInterval(() => {
      if (!pongReceived) {
        if (doc.conns.has(conn)) {
          closeConn(doc, conn);
        }
        clearInterval(pingInterval);
      } else if (doc.conns.has(conn)) {
        pongReceived = false;
        try {
          conn.ping();
        } catch (e) {
          closeConn(doc, conn);
          clearInterval(pingInterval);
        }
      }
    }, pingTimeout);

    // send sync step 1
    this.logger.debug(`Sending sync steps for document ${docId}`);
    send(doc, conn, encodeSyncStep1(Y.encodeStateVector(doc)))
    send(doc, conn, encodeSyncStep2(Y.encodeStateAsUpdate(doc)))
    if (doc.awareness.states.size > 0) {
      send(doc, conn, encodeAwarenessUpdate(doc.awareness, array.from(doc.awareness.states.keys())))
    }

    conn.on('close', () => {
      closeConn(doc, conn);
      clearInterval(pingInterval);
    });

    conn.on('pong', () => {
      pongReceived = true;
    });
  }

Thoughts

In most examples I have seen the document is stored in a persistend database (e.g. postgres) from the server. However, I would like to have the editor as a plugin. With the plugin-user having control and ability on where to store and potential expose different parts of the content.

By this point I am not sure if this is even possible and just an anti-pattern that should be avoided.

[Saul-Mirone]

IMO it's not possible. If your client can decide whether to sync it's content to server. Then maybe all clients are just cover contents from another and that's not controlled by CRDT.

[yswang0927]

@lexhuismans-wk You should use @hocuspocus/server as milkdown collab-server to save doc to database, see https://github.com/orgs/Milkdown/discussions/1756

But , milkdown does not fully support nodejs @Saul-Mirone

[Saul-Mirone]

Why should an editor support nodejs?

[yswang0927]

Why should an editor support nodejs?

To support saving both CRDT binary data and plain markdown content.

• save CRDT binary data to support collaboration.
• save plain markdown content to support reading it directly and rendering to the client (in non-editing mode).

As you said IMO it's not possible. If your client can decide whether to sync its content to server. Then maybe all clients are just cover contents from another and that's not controlled by CRDT. --- collaboration data must be saved at the server backend.

[Saul-Mirone]

If so, I think you can just install jsdom. Both milkdown and crepe run unit test with vitest in nodejs so I think it's not a blocker.

BTW, both the yjs binary and plain markdown is created and updated on client side. The server should only serve as a yjs provider and a database. The backend should not read or modify the data. That's how we did it in https://affine.pro.

[lexhuismans-wk]

Thanks! I just refactored everything to use the Hocuspocus backend together with the Redis extension, which looks very promising so far. I haven't seen a state where collaborative editing stopped working which is great.

Existing bugs

There is still a bug where sometimes the content will be duplicated on loading a document (I think this has to do with the timing of applying the template). This is an issue on my side.

And another bug where the listener plugin from Milkdown is not triggering onChange when another user sends updates, I think this is related to this discussion. Haven't figured out a workaround yet.