Merge 2025-04 RC Branch #345
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We need to regenerate the core OpenAPI modules for the upcoming `2025-04` release. - Regenerate core off of 2025-04. - Resolve a few minor TypeScript compile errors in the Assistant actions (we needed to use `as <Type>`). - This PR also includes a lot of integration test cleanup. Most of it is tweaking things to attempt to resolve the weird hangs we get from jest after successful runs. Primarily, I updated some of our error validation, upgraded jest deps to the latest version, and am now passing `--forceExit` when running tests with jest. There's follow up work to be done here to better clean up the integration suite in the client more generally. - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) CI - unit tests / integration tests / external app tests
## Problem
The new namespaces API operations are not implemented in the TypeScript
client, but they are available in the `2025-04` spec.
## Solution
- Add `/data/namespaces/` and new files defining actions:
`deleteNamespace`, `describeNamespace`, `listNamespaces`.
- Add `NamespacesOperationsProvider` to handle resolving index index
addresses and instantiating `NamespacesOperationsApi`.
- Update `Index` class to expose methods for namespaces operations.
- Add unit tests for `namespacesOperationsProvider`, and
`bulkOperationsProvider`.
- Add integration tests for namespaces operations.
- Regenerate off latest `2025-04` specifications.
- Update README.
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)
## Test Plan
CI: unit tests, integration tests, external app test
See the integration test file `namespaces.test.ts` for examples, or the
`README`:
```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('test-index');
// list all namespaces
const namespacesResp = await index.listNamespaces();
// describe a namespace
const namespace = await index.describeNamespace('ns1');
// delete a namespace (including all record data)
await index.deleteNamespace('ns1');
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1209571416923730
## Problem The Backups & Restore feature was added in the `2025-04` API, and needs to be implemented in the client. Documentation: https://docs.pinecone.io/guides/manage-data/backups-overview ## Solution Backups and Restore functionality adds a number of new operations / methods under the Pinecone class. Add new action functions and unit tests for the following: - `createBackup` - `createIndexFromBackup` - `describeBackup` - `describeRestoreJob` - `listBackups` - `listRestoreJobs` - `deleteBackup` This PR also updates the `README` to include documentation around the feature, along with doc comments inline for all new methods, types, etc. We skipped integration tests for this feature for the moment since it can take some time for a restoration job to complete. I've added detailed unit test coverage for basic piping between the interface and underlying OpenAPI internals, plus validation and error handling. I've also done a fair bit of manual testing which I'll share below: ``` > await client.createBackup({ indexName: 'local-utility-index', name: 'my-backup', description: 'this is my backup' }) { backupId: '3df86526-0edf-41b0-a555-513effa2146d', sourceIndexName: 'local-utility-index', sourceIndexId: '1d18673a-ed0e-4cdc-946f-541560367abf', name: 'my-backup', description: 'this is my backup', status: 'Initializing', cloud: 'aws', region: 'us-east-1', dimension: undefined, metric: undefined, recordCount: undefined, namespaceCount: undefined, sizeBytes: undefined, tags: {}, createdAt: '2025-05-08T14:52:00.236917882Z' } > await client.describeBackup('3df86526-0edf-41b0-a555-513effa2146d') { backupId: '3df86526-0edf-41b0-a555-513effa2146d', sourceIndexName: 'local-utility-index', sourceIndexId: '1d18673a-ed0e-4cdc-946f-541560367abf', name: 'my-backup', description: 'this is my backup', status: 'Ready', cloud: 'aws', region: 'us-east-1', dimension: 5, metric: undefined, recordCount: 0, namespaceCount: 2, sizeBytes: 66877, tags: {}, createdAt: '2025-05-08T14:52:00.236989Z' } > await client.listBackups() { data: [ { backupId: '3df86526-0edf-41b0-a555-513effa2146d', sourceIndexName: 'local-utility-index', sourceIndexId: '1d18673a-ed0e-4cdc-946f-541560367abf', name: 'my-backup', description: 'this is my backup', status: 'Ready', cloud: 'aws', region: 'us-east-1', dimension: 5, metric: undefined, recordCount: 0, namespaceCount: 2, sizeBytes: 66877, tags: {}, createdAt: '2025-05-08T14:52:00.236989Z' }, ... ], pagination: undefined } > await client.createIndexFromBackup({ backupId: '11450b9f-96e5-47e5-9186-03f346b1f385', name: 'my-restored-index' }) { restoreJobId: '3de68445-77bf-4a74-9790-440523b5213b', indexId: '8987dd9f-f3e4-49a3-bbd2-3d73cb9c2810' } > await client.listRestoreJobs() { data: [ { restoreJobId: '3de68445-77bf-4a74-9790-440523b5213b', backupId: '11450b9f-96e5-47e5-9186-03f346b1f385', targetIndexName: 'my-restored-index', targetIndexId: '8987dd9f-f3e4-49a3-bbd2-3d73cb9c2810', status: 'Pending', createdAt: 2025-05-08T14:55:12.749Z, completedAt: undefined, percentComplete: undefined }, ... ], pagination: undefined } ``` ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) ## Test Plan CI - external app test, integration tests, unit tests For manual testing, you can pull this branch down and run things locally with `npm run repl`. Set your `PINECONE_API_KEY` as an env variable, and then run the following: From the top level of the `pinecone-ts-client` repo: ```bash # this utility creates a new index and seeds some namespaces with data npx tsx ./utils/generateAndSeedIndex.ts npm run repl await init() await client.createBackup({ indexName: 'local-utility-index' }) await client.describeBackup('backup-id') await client.listBackups() await client.createIndexFromBackup({ backupId: 'backup-id' }) # etc ``` --- - To see the specific tasks where the Asana app for GitHub is being used, see below: - https://app.asana.com/0/0/1209571416750577
## Problem
Inference has new APIs which were made available in `2025-04`. These
allow for listing and describing available models hosted by Pinecone.
## Solution
- I took the liberty to refactor the `Inference` class to better align
with other classes used in a similar way such as `Index` and `Pinecone`.
I broke out all inference actions into individual files / functions:
`embed.ts` and `rerank.ts`. I think a lot of this was implemented rather
quickly initially, so I've done a bunch of cleanup while I was in here
adding new methods.
- Add new `getModel.ts` and `listModels.ts` files/functions. These are
called and documented inside `Inference`.
- Rework unit tests for `embed` and `rerank`. Basically just
standardized to how we do mocking / testing elsewhere. They were a bit
awkward.
- Add unit tests for `getModel`, `listModels`, along with an integration
test file.
- Update README to add examples working with models.
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)
## Test Plan
CI - external test, unit tests, integration tests
To test this you can pull this branch down, and run the repl locally:
from `pinecone-ts-client` root:
```
export PINECONE_API_KEY=<here>
npm run repl
await init()
await client.inference.listModels()
await client.inference.getModel('multilingual-e5-large')
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1209828518477622
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Changes for the
2025-04API release and associated features have been implemented in a set of PRs against the2025-04release candidate branch over the previous weeks:2025-04: Regenerate core from2025-04#341Solution
This PR merges all of these into
main. Testing and reviews have happened on each individual PR.Type of Change
Test Plan
CI - external apps, integration & unit tests