pg-boss getting started guide / examples

[brannan]

I've been programming on and off for a long time but somehow have never taken a deep dive into task queues. I did have one project that made extensive use of RabbitMQ, but I didn't do much of the work on that. So when it comes to pg-boss and task queues, I'm an idiot. I get the idea, certainly, but when to use send, publish, schedule took a little thought and digging around. Maybe this is the best way.

Your API docs are good, but but they miss the bigger picture for somebody who knows they need a task-queue but don't know the background theory and queuing terminology. For instance, "I want to offload the sending of messages to a queue. The messages will need to be sent at one specific time and date. Considerations and how to proceed?"

Anyway, I've dug through the docs, read a number of the discussions, and implemented the solution that I think I need. Well, it may not be best or even right, but it does work.

So I wrote a little "lay programmers guide to pg-boss" for my current project to document my understand of pg-boss. It's all of 57 lines long.

Would this be of any benefit to the project? Somebody who knows more than I do will have to review it. And clearly this is not a barn burner of an issue.

Perhaps just pulling a few more examples into the first readme would be a good idea. I'm happy to take a shot at this if anybody thinks it would be useful.

[timgit]

I agree with you that the Intro section of the docs is not a holistic intro to job queues, but rather how to use the npm package. This could definitely be improved.

I'd prefer to keep this very small, however, and maybe link to someone else's intro, perhaps something in the AWS SQS docs since I reference that on other places. Something that mentions asynchronicity, persistence, retries, and idempotency would be a good start.

[1mike12]

I cannot agree more with @brannan. I think this library may take 1st place for usefullness/impenetrable docs ratio for anything I've seen in the last few years. I know the library must be good with all the stars and great design, but without claude/gpt, I would have no idea how to interact with this without spending hours analyzing the source. Since so little is written about it, even the LLMs haven't had enough corpus to train on.

There is no other intro to this, except for an outdated article https://logsnag.com/blog/deep-dive-into-background-jobs-with-pg-boss-and-typescript

The responsibility of explaining how a library works to users lies with the library imo, not 3rd parties.

just a random list of questions that popped into my head within the last hour of getting started

Does this library take advantage of postgres' notify feature? I mean it's built specifically for pg v13+, there would be no way this is skipped for polling only right?

what does Raw SQL support for non-Node.js runtimes via INSERT or COPY mean?

it doesn't say how I have to setup my table for the library? Does it handle its own migration? what if I want to specify the table names?

what is the pubsub for and why/when would I use it vs the implicit calls?

how do I type the jobs? do I have to install a @typing ? does this have one?

What does the job object look like?

why is the job to be processed an array?

[infovore]

+1. @brannan , did you ever publish those 57 lines anywhere? I'd be grateful to see even somebody else's approach.

I both understand the sample code and cannot see, without having to do a lot of hammering myself, how to apply it in anything larger than a toy scenario. Should I, for instance, be always hitting the 'create queue' function in my worker script, and hope it just silently does nothing if the queue/tables already exist?

The sample code proves how things work, and doesn't really indicate anything about how to use it; at the very least, a sample of "separating out worker processing from pushing onto the queue" would be useful, given... that's mainly how you use a queue.

I don't need a holistic introduction to queues. I do, however, need more than API docs - demonstrating this library's preferred approach to common problems, its own philosophy explained. (The last thing I want, ever, is a link to AWS documentation, which I almost universally loathe).