Skip to content

Players tab errors out if >65536 players are in database #1091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Jovvik opened this issue Apr 2, 2025 · 0 comments
Closed

Players tab errors out if >65536 players are in database #1091

Jovvik opened this issue Apr 2, 2025 · 0 comments
Labels
bug next-release-resolved This issue will be solved in the next release

Comments

@Jovvik
Copy link

Jovvik commented Apr 2, 2025

Describe the bug

If demos with more than 65536 players in total are imported, the players tab fails to load with "An error occured.". Does not happen if filters are applied such that only <64K players are returned.

Image

Demo links

No response

Steps to reproduce

  1. Ingest enough demos
  2. Try to open the players tab

Expected behavior

I would expect the players tab not to error out, but it does

How often does this bug happen?

None

Log file

Wed, 02 Apr 2025 14:03:25 GMT | ERROR | server | Error while fetching players table
Wed, 02 Apr 2025 14:03:25 GMT | ERROR | server | error: bind message has 62534 parameter formats but 0 parameters
    at /home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/pg/lib/client.js:535:17
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at NY.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/dialect/postgres/postgres-driver.js:72:28)
    at NY.e.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/driver/runtime-driver.js:90:24)
    at /home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:35:28
    at yD.provideConnection (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/driver/default-connection-provider.js:10:20)
    at t.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:34:16)
    at t.execute (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-builder/select-query-builder.js:315:24)
    at NJt (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/node/database/players/fetch-players-table.ts:124:27)
    at async Promise.all (index 0)
    at lPe (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/node/database/players/fetch-players-table.ts:200:35)
    at cPe (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/server/handlers/renderer-process/player/fetch-players-table-handler.ts:7:21)
    at t.onRendererProcessSocketMessage (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/server/server.ts:175:26)
    at NY.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/dialect/postgres/postgres-driver.js:92:41)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at NY.e.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/driver/runtime-driver.js:90:24)
    at /home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:35:28
    at yD.provideConnection (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/driver/default-connection-provider.js:10:20)
    at t.executeQuery (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:34:16)
    at t.execute (/home/snek/cs-demo-manager/CS Demo Manager/resources/node_modules/kysely/dist/esm/query-builder/select-query-builder.js:315:24)
    at NJt (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/node/database/players/fetch-players-table.ts:124:27)
    at async Promise.all (index 0)
    at lPe (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/node/database/players/fetch-players-table.ts:200:35)
    at cPe (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/server/handlers/renderer-process/player/fetch-players-table-handler.ts:7:21)
    at t.onRendererProcessSocketMessage (/home/snek/cs-demo-manager/CS Demo Manager/resources/src/server/server.ts:175:26)
Wed, 02 Apr 2025 14:03:25 GMT | ERROR | server | WS:: error handling message with fetch-players-table from renderer process
Wed, 02 Apr 2025 14:03:25 GMT | ERROR | server | 1
Wed, 02 Apr 2025 14:03:26 GMT | LOG | server | demo analyzer exited with code 0

Environment information

Version: 3.12.0
OS: linux x64 6.13.8-zen1-1-zen
Electron: 34.3.0
Chrome: 132.0.6834.210
Last database migrations:
v7 - 2025-03-19T11:57:43.630Z

Additional context

The error is due to

.where('players.steam_id', 'in', steamIds)

When steamIds contains >64K values, node-postgres errors out, because postgres only supports up to 65536 parameters per a query. steamIds should probably be split into 64K-sized chunks to avoid this.

Here's a comment that explains the error message: brianc/node-postgres#2579 (comment)

@akiver akiver added the bug label Apr 6, 2025
@akiver akiver added the next-release-resolved This issue will be solved in the next release label Apr 8, 2025
@akiver akiver closed this as completed May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug next-release-resolved This issue will be solved in the next release
Projects
None yet
Development

No branches or pull requests

2 participants