Inquiry About the register_client Method in Your Library

[rekvizitt]

Hello,

I am currently using your library and would like to gain a better understanding of the register_client method. I have a few questions regarding its functionality:

1. How does the register_client method work? What parameters does it accept, and what happens after it is called?
2. Are there any specific features or limitations I should be aware of when using this method?
3. Could you provide an example of how to use register_client in the context of working with WebSocket?

Thank you in advance for your assistance!

[dosubot[bot]]

Hello @rekvizitt! I'm here to help you with any questions you have about the library. Let's tackle this together!

1. The register_client method is used to register a new client connection to a WebSocket server. It accepts a single parameter: websocket, which is an instance of WebSocketServerProtocol. After it is called, the client connection is added to the connected_clients set, and a log message is generated to indicate the new connection. The method then enters a loop to handle messages from the client, although the current implementation does not process any messages. If the connection is closed normally, a log message is generated, and the client is removed from the connected_clients set ^[1].

2. There are no specific features or limitations documented in the codebase regarding the register_client method ^[1].

3. An example of using the register_client method with WebSocket is provided in the codebase. The method is defined in the TiktokWebSocketCrawler class and is used to handle new client connections, log their connection, and manage their lifecycle within the WebSocket server. Here is a relevant code snippet:

   async def register_client(self, websocket: WebSocketServerProtocol) -> None:
       """
       注册新的客户端连接
   
       Args:
           websocket: WebSocketServerProtocol 实例
       """
   
       self.connected_clients.add(websocket)
       logger.info(
           _("[RegisterClient] [🔗 新的客户端连接] ｜ [Ip：{0} Port：{1}]").format(
               *websocket.remote_address
           )
       )
       try:
           async for message in websocket:
               # TODO: 处理客户端消息或鉴权
               pass
       except ConnectionClosedOK:
           logger.info(
               _("[RegisterClient] [⛓ 客户端断开连接] | [Ip：{0} Port：{1}]").format(
                   *websocket.remote_address
               )
           )
       finally:
           self.connected_clients.remove(websocket)

   This method is used to handle new client connections, log their connection, and manage their lifecycle within the WebSocket server ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[rekvizitt]

@dosu,
here is a sample code where I use fetch_live_danmaku.

import asyncio
from api.tiktok.handler import TiktokHandler
import websockets

async def main():
    kwargs = {
        "headers": {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
            "Referer": "https://www.tiktok.com/",
        },
        "timeout": 10,
        "proxies": {
            "http://": None,
            "https://": None,
        },
        "cookie": "",
    }
    handler = TiktokHandler(kwargs)
    uniqueId = "uniqueId"
    
    live_info = await handler.fetch_user_live_videos(uniqueId)
    room_id = live_info.live_room_id

    live_im_info = await handler.fetch_live_im(room_id)
    internal_ext = live_im_info.internalExt
    cursor = live_im_info.cursor
    wrss = live_im_info.routeParams.wrss
    
    websocket_live = await handler.fetch_live_danmaku(room_id, internal_ext, cursor, wrss)
    # # TODO: create a websocket and register_client
    # client_websocket = "localhost"
    # websocket_live.register_client(client_websocket)
    
if __name__ == "__main__":
    asyncio.run(main())

I don't understand which websocket server I should use. Besides that I don't understand how to do it exactly.

[dosubot[bot]]

To use the fetch_live_danmaku method effectively, you need to set up a WebSocket server that can handle client connections. The DouyinWebSocketCrawler class in the f2 library provides a WebSocket server implementation that you can use. Here's how you can proceed:

1. Set Up a WebSocket Server: The DouyinWebSocketCrawler class has a start_server method that starts a WebSocket server. You can use this method to handle incoming WebSocket connections.

2. Register Clients: Use the register_client method to manage client connections. This method is responsible for adding new clients to the server and handling their lifecycle.

Here's a modified version of your code to illustrate how you can set up and use the WebSocket server:

import asyncio
from api.tiktok.handler import TiktokHandler
from f2.apps.douyin.crawler import DouyinWebSocketCrawler

async def main():
    kwargs = {
        "headers": {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
            "Referer": "https://www.tiktok.com/",
        },
        "timeout": 10,
        "proxies": {
            "http://": None,
            "https://": None,
        },
        "cookie": "",
    }
    handler = TiktokHandler(kwargs)
    uniqueId = "uniqueId"
    
    live_info = await handler.fetch_user_live_videos(uniqueId)
    room_id = live_info.live_room_id

    live_im_info = await handler.fetch_live_im(room_id)
    internal_ext = live_im_info.internalExt
    cursor = live_im_info.cursor
    wrss = live_im_info.routeParams.wrss
    
    # Start the WebSocket server
    websocket_crawler = DouyinWebSocketCrawler()
    await websocket_crawler.start_server()

    # Fetch live danmaku
    await handler.fetch_live_danmaku(room_id, internal_ext, cursor, wrss)

if __name__ == "__main__":
    asyncio.run(main())

In this setup, the DouyinWebSocketCrawler class is used to start a WebSocket server. The fetch_live_danmaku method will connect to the WebSocket endpoint and start receiving messages, which can be handled by the server ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[Johnserf-Seed]

Hello! Thank you so much for using our library and diving deep into the technical details. Below are the clarifications to your questions:

---

1. Functionality of register_client

• Purpose: register_client is an internal interface and should not be called directly by users.
• Behavior:
  • Automatically manages client connections by adding/removing them from connected_clients.
  • Logs connection/disconnection events.
  • Enables message broadcasting to all connected clients.

2. Usage Guidance

• No Action Required: Simply follow the example in docs/snippets/tiktok/user-live-im-fetch.py.
• Workflow: The framework handles client registration internally once the local server starts.

3. Current Limitations

1. Authentication & Bidirectional Communication
   • No authentication for client connections (planned in TODO).
   • Clients cannot send control commands (e.g., switching callbacks).
2. Ephemeral Data Storage
   • connected_clients is an in-memory collection – data is lost on server restart.
   • Extend with persistent storage (e.g., Redis) if needed.
3. Concurrency Considerations
   • Add thread locks for high-frequency connections:

from threading import Lock
class TiktokWebSocketCrawler:
    def __init__(self):
        ......
        self.lock = Lock()
        self.connected_clients = set()
        ......

    async def register_client(self, websocket):
        with self.lock:
            self.connected_clients.add(websocket)

4. Future Roadmap

• Authentication: Implement token validation via a pre_connect hook.
• Bidirectional Communication: Allow clients to send control commands.
• Cluster Support: Add Redis adapter for multi-instance synchronization.

Feel free to reach out for further discussions! 🚀

---

ps. The endpoints and ports for ws are configured in conf.yaml

[rekvizitt]

Thank you so much!

[rekvizitt]

local websocket closes automatically after 10 seconds.

websocket closes the connection because of this function.

    async def _timeout_check(self, server: WebSocketServer) -> None:
        “"”
        Checks the timeout for the server.

        Args:
            server: WebSocketServer object.
        “"”
        While True:
            await asyncio.sleep(self.timeout)
            if not self.connected_clients:
                logger.info(
                    (
                        “[TimeoutCheck] [⏳ Closing due to no clients] | [Timeout: {0} sec]”
                    ).format(self.timeout)
                )
                break
        server.close()
        await self.close_websocket()
        

---

The point was to register a local websocket?
here:

server = await serve(self.register_client, wss_domain, wss_port)

[Johnserf-Seed]

Yes, this is to save performance on the local server, the local client needs to connect within 10 seconds (especially important in multiple live streams with lots of popup messages). You can modify the timeout in the config file or customize the timeout when building the kwargs2 parameter, if you don't need the timeout setting you can modify the source code to block it.

[rekvizitt]

Okay, now I know exactly why it's done that way. That's smart, thanks for the reply!