doc: Add a repo-level README

[anubhav756]

This PR includes adding a top-level README for the repo.

The README focusses on an overall guidance of using the subpackages included in this mono-repo.

Note

This README would be simplified, especially the code snippets, once we align the APIs of the toolbox-core and orchestration-specific SDKs, like toolbox-langchain, in a future PR.

[anubhav756]

/gcbrun

[twishabansal]

We need to make sure our interfaces in toolbox-core and toolbox-langchain align.

[anubhav756]

We need to make sure our interfaces in toolbox-core and toolbox-langchain align.

As discussed offline, we will be doing the breaking changes in an upcoming PR.

[twishabansal]

Thinking about the main README, I'm wondering if the number of toolbox-langchain code snippets is becoming a maintenance burden. To avoid having to update code examples in multiple places for every small change, perhaps we could reduce the snippets in the main README and link to the toolbox-langchain README for more detailed examples? What are your thoughts?

[anubhav756]

Thinking about the main README, I'm wondering if the number of toolbox-langchain code snippets is becoming a maintenance burden. To avoid having to update code examples in multiple places for every small change, perhaps we could reduce the snippets in the main README and link to the toolbox-langchain README for more detailed examples? What are your thoughts?

I've kept them here for now, anticipating the upcoming API convergence between the Core and Orch SDKs (see doc). This convergence should allow us to easily showcase consistent snippets across both of these subpackages (along with the upcoming subpackage). Given this expected future state of reduced maintenance due to API alignment, do you think it's best to keep them here temporarily?

[twishabansal]

Thinking about the main README, I'm wondering if the number of toolbox-langchain code snippets is becoming a maintenance burden. To avoid having to update code examples in multiple places for every small change, perhaps we could reduce the snippets in the main README and link to the toolbox-langchain README for more detailed examples? What are your thoughts?

I've kept them here for now, anticipating the upcoming API convergence between the Core and Orch SDKs (see doc). This convergence should allow us to easily showcase consistent snippets across both of these subpackages (along with the upcoming subpackage). Given this expected future state of reduced maintenance due to API alignment, do you think it's best to keep them here temporarily?

We could remove these once the APIs are aligned. Sounds good. Thanks!

[kurtisvg]

Should we replicate the contents of the other two READMEs here?

Or should e focus which packages are in these repo, and let the top level README try to answer questions like "which package should I use in which circumstances"?

[anubhav756]

...should e focus which packages are in these repo, and let the top level README try to answer questions like "which package should I use in which circumstances"?

I think the current draft already covers this (sections "Available Packages", "Usage Comparison", and also "Synchronous vs. Asynchronous Usage"). Could you let me know if you feel something specific is missing or unclear on that front?

Should we replicate the contents of the other two READMEs here?

I don't have strong opinions against simply listing packages. However, I was thinking about the longer-term picture, especially post-API convergence. Ideally, the core usage across different packages will become very similar, with consistent function signatures and documentation.

With that in mind, I was leaning towards leveraging the repo-level README to showcase common functionalities upfront using code snippets. This could help explain core Toolbox concepts like Bound Params and Auth Tokens more clearly and directly, rather than linking out to multiple package-specific READMEs for these shared features. My concern with just linking out is that it might become distracting for someone trying to get a high-level overview and could inadvertently give the impression that the packages are more different in their core usage than they actually are (or will be).

How about this approach: The repo-level README focuses on the common workflow? We could structure it to say, "Choose and import the right package based on [guidance already provided], and then the core operations like binding parameters or configuring auth tokens work like this..." followed by the common code examples.

We would definitely still keep the detailed, tailor-made READMEs within each subpackage for any package-specific nuances or setup. But for explaining and illustrating the Toolbox-specific features (as opposed to orchestration features) that span across packages, like bound params and auth tokens, the repo-level README feels like a good place to consolidate that information and the corresponding code examples.

A significant benefit I see is maintainability: if we add or change a common Toolbox feature across the SDKs in the future, we'd primarily update the documentation and examples in this single, central README.

What are your thoughts on this direction?

[kurtisvg]

I want us to optimize for the intersection of clarity and maintainability.

My current thought process is that we really need to communicate 2 things to users:

1. which package they should use.
2. how to use the package

It's obviously more

For the first thing, I think the top-level README has to address it. Hopefully if they are at the package README they already known, but the package level READMEs should have a little INFO box that says something like "This is the <> package, intended to be used with <>. If you aren't sure which package to use, see the project README.".

For the second thing, we have a common functionality (of across all packages), and package specific functionality. I think we have to include the "how-to" for package-level functionality. Additionally, the READMEs are also present on the pypi package downloads. Given that, I think we should include all the "how-to" (for both common and specific) in the package level READMEs because it's more clear.

However, I lean towards avoid putting the common functionality in the top-level README. It's both a second place to keep up to date, and it's less clear because a user may not read in it's entirely and miss the link to the package specific readme.

TL;DR: My 2c is that the intersection of clarity/maintainability is that top level README should cover "which package" and the package level README should cover "how to use".

[anubhav756]

I want us to optimize for the intersection of clarity and maintainability.

My current thought process is that we really need to communicate 2 things to users:

1. which package they should use.
2. how to use the package

It's obviously more

For the first thing, I think the top-level README has to address it. Hopefully if they are at the package README they already known, but the package level READMEs should have a little INFO box that says something like "This is the <> package, intended to be used with <>. If you aren't sure which package to use, see the project README.".

For the second thing, we have a common functionality (of across all packages), and package specific functionality. I think we have to include the "how-to" for package-level functionality. Additionally, the READMEs are also present on the pypi package downloads. Given that, I think we should include all the "how-to" (for both common and specific) in the package level READMEs because it's more clear.

However, I lean towards avoid putting the common functionality in the top-level README. It's both a second place to keep up to date, and it's less clear because a user may not read in it's entirely and miss the link to the package specific readme.

TL;DR: My 2c is that the intersection of clarity/maintainability is that top level README should cover "which package" and the package level README should cover "how to use".

Totally agree with your reasoning – if I think your way, then it makes more sense to have everything needed for a package only in its dedicated README for clarity. Even if we repeat some common setup info across the package READMEs, the potential gains from trying to maintain a single, separate repo-level 'how-to' seem pretty small compared to the confusion it could cause for users.

Just updated the main repo README:

• It's now focused just on helping users figure out which package they need (core vs orch-specific).
• Removed the detailed how-to sections (like core concepts, etc.) to keep it clean. Those details belong in the package READMEs now.
• Added a specific "Which Package Should I Use?" section upfront.
• Simplified the "Getting Started" > "Use the SDK" part - it just points to the "Available Packages" list for the specific README links now, so we don't have to repeat them (scales better with more packages! 👍).

Let me know how it looks now 🙂