Is Node viable in its current way of development and maintenance?

[gkatsanos]

UPDATE: Please dont focus on fetch, this is not the reason for this thread, it's supposed to be more of a birds-eye view on how the project is going, point to useful Public Working Groups, Roadmaps etc.

I would like to open a no-agenda thread which was epidermically discussed in nodejs/node#19393 .

With the current number of active maintainers, the current project management (if any), the current funds and sponsorships, is Node a project where the community and maintainers, as well as potential corporations interested to use it, a project which has a future (adoption rate, new apis, maintenance), or are there changes needed in order to improve both the workload of the maintainers, as well as the quality of the API, or tackle the 950+ open issues in the issue queue? How does Node compare with the release cycle of other server/frameworks?

[benjamingr]

I mean, I think Node.js still has very high velocity IMO with multiple (large) things shipped in the last year (cancellation in all APIs, QUIC, policies, unflagging ESM, promise UX improvements, big native API changes, new timer promises APIs, worker improvements, making core tamper safe with primordials sourcemaps and a ton more).

It's just that people here don't particularly care that much about fetch in core enough to invest a ton of time in doing it right. It's still progressing. The 955 open issues compare to over 11000 closed ones, ditto for PRs.

[benjamingr]

How does Node compare with the release cycle of other server/frameworks?

And Node has much higher velocity and much finer grained releases with release lines compared to most other platforms I'm aware of. There is a lot that can improve and a lot Node.js can do better.

You are welcome to get involved in the "Next 10 years" initiative for example :]

[gkatsanos]

How does Node compare with the release cycle of other server/frameworks?

You are welcome to get involved in the "Next 10 years" initiative for example :]

See this is why it's good to bring these things to the surface, I didn't even know this existed - I just discovered the repo. Isn't it invite only though?

[Trott]

The short answer is (IMO) "Yes, broadly speaking, the current way of development and maintenance is more than viable."

Of course, there are areas for improvement. That will always be true.

I think the fetch discussion and a few other things could benefit from a bit of a more formal process around it and there's some discussion of that in nodejs/TSC#962.

I don't think our velocity is slow at all, especially considering the size of our user base, diversity of supported platforms, and scope/variety of use cases in the community. Our release pace and support cycle is something we do well.

And while I'd love to see the number of languishing open issues and pull requests decrease, I don't find the number of open issues and pull requests to be a cause for any significant concern about the viability of the project, especially when compared to other projects that have similarly large user bases and/or code bases.

Python currently has 1440 open pull requests compared to our 251.

tensorflow has 3773 open issues compared to our 979.

redis has significantly more open issues and pull requests than we do.

PowerShell has approximately 3000 open issues.

aspnetcore has approximately 2500 open issues.

electron has approximately 1500 open issues.

react-native has more open issues and more open pull requests than we do.

And so on. In terms of the number of open issues and pull requests being a source of 😱, I'd say we can and should try to do better. But in terms of it suggesting a threat to our viability, counterexamples are everywhere.

That said, it's frustrating for people to have their issues and pull requests languish. Aside from the RFC issue linked above, I think our most recent experiment in addressing this is the creation of a Triagers team. If curious/interested, see https://github.com/nodejs/node/blob/master/doc/guides/contributing/issues.md#triaging-a-bug-report for information on how that works, how to join, etc.

[Trott]

See this is why it's good to bring these things to the surface, I didn't even know this existed - I just discovered the repo.

Repository fragmentation is an issue we need to address. There are good reasons for it in a lot of cases, but it hampers discoverability of information.

[benjamingr]

See this is why it's good to bring these things to the surface, I didn't even know this existed - I just discovered the repo. Isn't it invite only though?

You are welcome to open an issue to ask how to engage - and ask to be included. There is a good chance you'll be able to contribute. Not sure though - pinging @nodejs/next-10 -> is the initiative currently only open to people inside the org?

[gkatsanos]

Thanks, this thread is already bringing more info than I hoped :) looking forward for more insights. I think a good way to start contributing is by answering pure support questions in the issue tracker.

[mcollina]

Regarding fetch, the reason why fetch has not happened in core yet is because node-fetch gives 80% of the feature needed. From talking to companies with budget to invest in this, there is no business case in adding fetch to core.

As for the rest, very few have the interest in adding something non-spec compliant to core that we'd have to maintain for a long time. It'd be hard work that few volunteers will be willing to take on.

If you are interested in what next-gen http clients will look like, take a look at https://github.com/nodejs/undici - it's likely going to get a fetch-like implementation soon from @Ethan-Arrowood.

[gkatsanos]

update from my side, just discovered some of the Node working group meetings are on Youtube: https://www.youtube.com/c/nodejs-foundation/videos

[Qard]

Expanding on what @mcollina said, fetch might look simple on the surface, but following the spec there's a bunch more depth than is immediately obvious, with a bunch of different interconnected types that also need to be implemented to follow the script exactly. Some of those have been introduced recently, such as AbortController and Blob, but there are many more yet which have not like FormData, and some of which are in direct conflict with core APIs like ReadableStream (The ecma spec streams, not Node.js streams). I would think it's not impossible that core may eventually gain fetch support, but there are many prerequisites and someone needs to build those first.

[jasnell]

@Trott:

I think the fetch discussion and a few other things could benefit from a bit of a more formal process around it and there's some discussion of that in nodejs/TSC#962.

I have to disgree there. I don't think we need more formal process at this point. It would be pointless. The reason we don't have XYZ (where XYZ is whatever feature someone may want) is that existing contributors have not prioritized doing it and new contributors have not yet contributed it. Adding formal process won't change that fact.

[benjamingr]

but there are many more yet which have not like Blob

Fwiw blob landed nodejs/node#36811 🎉

[Qard]

Ah, missed that one. Cool. 🎉

[jasnell]

Well, let's say partial blob support landed. I'm still working on the async blob backend.

[Trott]

The reason we don't have XYZ (where XYZ is whatever feature someone may want) is that existing contributors have not prioritized doing it and new contributors have not yet contributed it. Adding formal process won't change that fact.

@jasnell Maybe fetch is no longer a good example because consensus has gelled. But I think it still applies to arbitrary feature XYZ.

By "formal", I don't mean "heavyweight". I mean a clear and efficient mechanism for Node.js as a project to say "Yes, that's a feature we want", or "No, we would not welcome such a feature." We have a mechanism for these things. It's not efficient. And it burns people out. (That's the problem a lightweight decision-making mechanism would be trying to solve, not the problem of "Why don't we have feature XYZ implemented yet?")

I'll make the counterpoint argument to what I'm saying, just to save others the trouble: It's possible that long and often-frustrating conversations are necessary to make these decisions. There's a lot to consider sometimes and you have to talk it over, sometimes multiple times.

I hope there's a not-onerous way to reduce the friction. If other projects/organizations have found a solution that works, perhaps we should consider what they do. Perhaps the reason we have punishingly long conversations around certain things is exactly because we have no good process for making decisions around whether or not features should be implemented.

[gkatsanos]

I'll make the counterpoint argument to what I'm saying, just to save others the trouble: It's possible that long and often-frustrating conversations are necessary to make these decisions. There's a lot to consider sometimes and you have to talk it over, sometimes multiple times.

I'll go ahead an speculate that developer-only communities who work 100% remotely and by default do not proactively communicate will tend to develop a series of collaboration issues that we see and trying to solve in our day jobs for the past year or so.. That relates to your point on lack of "long and often frustrating conversations". I can imagine for a volunteer who just wants to jump in, make commits, and go back to their lives, having to have a one hour - long - unpaid call with other developers are worse having to pass his status to a PM or a PO isn't the most pleasant thing in the world.

But that also results in what we saw in the other thread. The left hand didn't know what the right hand was doing. The thread was up for months (years?) and only then someone who knew exactly the dependencies and status of the pre-reqs for something was able to enlighten the others.

[mcollina]

But that also results in what we saw in the other thread. The left hand didn't know what the right hand was doing. The thread was up for months (years?) and only then someone who knew exactly the dependencies and status of the pre-reqs for something was able to enlighten the others.

I think the problem is slightly different - most of the contributors knew the situation as we talked so many times at the various collaborators summits and online. However these are not really surfaced to our users, for which node.js looks a lot like a black box. In the past we did a few updates on modules / ESM (https://medium.com/the-node-js-collection/the-current-state-of-implementation-and-planning-for-esmodules-a4ecb2aac07a) that informed the community on how things were progressing. I think we should do more blog posts on that - ideally one on fetch.

Note that most of the time when an issue became too "hot", with people doing +1s and not participating in the discussion, many people in the collaborators group stop engaging.

[mhdawson]

@gkatsanos you are free to join the next-10 discussions. The meetings are on the calendar: https://calendar.google.com/calendar/u/0/embed?src=nodejs.org_nr77ama8p7d7f9ajrpnu506c98@group.calendar.google.com

The next one is Feb 25 at 12 ET, there will be an issue created in the repo a few days before in https://github.com/nodejs/next-10/issues with the details for joining the call.

[mhdawson]

I agree with @mcollina that having better organization/communication on larger feature requests would be good for end users. I think it is related to the comments from @Trott with respect to a process. If we had an agreed way that we would:

• Identify/document major features (examples might include ES6 modules, fetch) for which end users have exceptions
• Come to an agreement on whether it's a must/should/like/would tolerate/should not to be included in Node.js
• Document current status, for example evaluating, actively being worked, waiting for volunteers etc.
• Communicate the above.

Maybe just having a place to document the list from the first bullet point would be start and we could fill in the later pieces even if the discussions around the features take place in the same way they do today.

[mcollina]

I think just doing the first point would be enough.

[Qard]

Most of the time when any given feature has a lot of demand but isn't immediately coming into reality, it's because it is more complex than may be immediately obvious. In the case of fetch, it's a relatively simple surface area but with a lot more type dependency depth than a user might realize.

The problem is that we often outright dismiss features like that due to the complexity rather than communicating that feature x depends on the existence of features y and z. If we more clearly express what the blockers are for something being able to exist in core then there's a much clearer path for people with an interest in that feature to be able to move things gradually forward to a place where it might be possible to have.

Node.js doesn't really have a project manager to lay out a roadmap that contributors can gradually work towards, we just have a bunch of people working in relative isolation and dropping a bunch of code on the project. Sometimes effort overlaps, sometimes it's even in conflict, because we don't really communicate roadmap well or highlight who is working on what.

[gkatsanos]

@gkatsanos you are free to join the next-10 discussions. The meetings are on the calendar: https://calendar.google.com/calendar/u/0/embed?src=nodejs.org_nr77ama8p7d7f9ajrpnu506c98@group.calendar.google.com

The next one is Feb 25 at 12 ET, there will be an issue created in the repo a few days before in https://github.com/nodejs/next-10/issues with the details for joining the call.

Thank you! I appreciate all the useful insights and information posted so far and I'll definitely seize the opportunity to attend :)

[splitice]

@benjamingr FYI QUIC was reversed.

[jasnell]

@splitice ... That's temporary hopefully

[bnb]

Want to comment since I don't see an answer to an explicitly asked question: AFAIK like almost all teams, WGs, and Initiatives in Node.js, the Next 10 work is open for anyone to join.

Edit: oops, I skimmed too hard and didn't see @mhdawson's reply 😅