Skip to content

Conversation

dylburger
Copy link
Contributor

@dylburger dylburger commented Sep 26, 2024

WHY

Summary by CodeRabbit

  • New Features

    • Introduced a new BackendClient class with updated methods for improved API interactions.
    • Added functionality to invoke workflows for external users.
    • Enhanced client creation process with clearer parameter structures.
    • Streamlined app connection logic with updated app identifier types.
    • Added support for OAuth credentials in the client configuration.
  • Bug Fixes

    • Enhanced error handling and validation in the client methods.
  • Documentation

    • Updated README with instructions for importing the SDK in different environments.
    • Introduced detailed contributing guidelines for local development.
  • Chores

    • Specified Node.js version for development environment.
    • Updated package version to 1.0.0.
    • Added a GitHub Actions workflow for automatic Markdown linting.
    • Updated GitHub Actions workflow for running SDK tests with improved concurrency settings.

Copy link

vercel bot commented Sep 26, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
docs-v2 ⬜️ Ignored (Inspect) Visit Preview Nov 4, 2024 7:45pm
pipedream-docs ⬜️ Ignored (Inspect) Nov 4, 2024 7:45pm
pipedream-docs-redirect-do-not-edit ⬜️ Ignored (Inspect) Nov 4, 2024 7:45pm

Copy link
Contributor

coderabbitai bot commented Sep 26, 2024

Walkthrough

The changes in this pull request involve a comprehensive update to the SDK, specifically transitioning from the ServerClient class to a new BackendClient class. Key modifications include renaming methods, updating method signatures to accommodate OAuth parameters, and introducing new functionality for workflow invocation. The test suite has also been revised to reflect these changes, ensuring thorough coverage of the new client structure and error handling scenarios. Additionally, the README has been updated with development instructions, and a specific Node.js version has been set for the project.

Changes

File Path Change Summary
packages/sdk/src/server/__tests__/server.test.ts - Transitioned test suite to BackendClient class.
- Updated method signatures and added new test cases for invokeWorkflowForExternalUser and buildWorkflowUrl.
- Renamed various methods for consistency.
packages/sdk/src/server/index.ts - Renamed CreateServerClientOpts to BackendClientOpts and updated properties.
- Introduced OAuthCredentials type and HTTPAuthType enum.
- Added invokeWorkflowForExternalUser method.
- Renamed methods for clarity.
packages/sdk/src/browser/index.ts - Replaced AppId with AppNameSlug and removed StartConnectApp.
- Updated StartConnectOpts type.
- Renamed createClient to createFrontendClient with updated signature.
packages/sdk/README.md - Added "Importing the Client" section with instructions for importing the SDK in both CommonJS and ES module formats.
packages/sdk/.tool-versions - Added Node.js version 22.10.0 for environment configuration.
packages/sdk/package.json - Updated version from 0.1.9 to 1.0.0.
- Simplified browser field.
- Updated exports section.
- Added watch script and nodemon as a development dependency.
packages/sdk/CONTRIBUTING.md - Added "How to Contribute to the SDK" section with instructions for setting up a local development environment, installing dependencies, and building the package.
packages/sdk/CHANGELOG.md - Introduced version 1.0.0, detailing significant modifications to the server-side client class and authentication mechanism.
packages/sdk/tsconfig.node.json - Updated compilerOptions, include, and exclude properties for better compilation management.
packages/sdk/jest.setup.js - Mocked the simple-oauth2 module for testing purposes.
packages/sdk/.tool-versions - Specified Node.js version 22.10.0 for development.

Possibly related PRs

Poem

🐇 In the land of code where changes bloom,
A BackendClient rises, dispelling the gloom.
OAuth flows in, with methods anew,
Testing and building, all clear in view.
With each little tweak, our SDK shines bright,
Hopping along, ready for flight! ✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Outside diff range and nitpick comments (5)
packages/sdk/src/server/__tests__/server.test.ts (2)

551-554: Inconsistent secretKey value in test initialization

In this test setup, the secretKey is set to "test", whereas in other tests it's set to "test-secret" or "test-secret-key". For consistency and to avoid confusion, consider using a uniform mock value for secretKey across all tests.


626-629: Inconsistent secretKey value in test initialization

Similar to a previous test, the secretKey here is set to "test" instead of a consistent value like "test-secret-key". Aligning mock values across tests improves readability and reduces potential confusion.

packages/sdk/src/server/index.ts (3)

10-13: Add JSDoc comments for the ProjectKeys type

Consider adding JSDoc comments to the ProjectKeys type to describe its purpose and usage. This will enhance code readability and help other developers understand how to use this type effectively.


15-18: Add JSDoc comments for the PipedreamOAuthClient type

Adding JSDoc comments to the PipedreamOAuthClient type will improve maintainability and provide clarity on its intended use within the SDK.


842-844: Enhance error message for missing project public key

The error message when this.publicKey is not set could be more informative:

if (!this.publicKey) {
  throw new Error("Project public key is required to map the external user ID to the correct project");
}

Consider providing guidance on how to supply the publicKey, helping developers resolve the issue quickly.

Update the error message to:

 throw new Error(
-  "Project public key is required to map the external user ID to the correct project"
+  "Project public key is required. Please provide it via the 'project' option in 'CreateServerClientOpts'."
 );
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 2dc725e and 6d24e66.

📒 Files selected for processing (2)
  • packages/sdk/src/server/tests/server.test.ts (4 hunks)
  • packages/sdk/src/server/index.ts (4 hunks)
🔇 Additional comments not posted (2)
packages/sdk/src/server/__tests__/server.test.ts (2)

667-762: Comprehensive tests for invokeWorkflowForExternalUser

The new test suite for invokeWorkflowForExternalUser thoroughly covers expected functionality and error handling scenarios, including header inclusion and validation of required parameters. This enhances the reliability of the code.


25-35: Verify all usages of createClient are updated to the new signature

While this test confirms that createClient works with the new project parameter structure, it's important to ensure that all instances of createClient in the codebase have been updated to match this new signature to prevent runtime errors.

Run the following script to find usages of createClient that may need updating:

✅ Verification successful

All usages of createClient have been verified to use the new project parameter structure.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find all usages of `createClient` not using the new `project` parameter structure.

# Test: Search for calls to `createClient` that don't include the `project` object.

rg --type typescript --pcre2 'createClient\(\s*\{(?!.*project\s*:).*\}\s*\)'

Length of output: 113


Script:

#!/bin/bash
# Description: Find all usages of `createClient` not using the new `project` parameter structure.

# Corrected Test: Search for calls to `createClient` that don't include the `project` object.

rg --glob '*.ts' --glob '*.tsx' --pcre2 'createClient\(\s*\{(?!.*project\s*:).*\}\s*\)'

Length of output: 89

Comment on lines 549 to 561
const client = new ServerClient(
{
publicKey: "test-public-key",
secretKey: "test-secret-key",
oauthClientId: "test-client-id",
oauthClientSecret: "test-client-secret",
project: {
publicKey: "test-public-key",
secretKey: "test",
},
oauth: {
clientId: "test-client-id",
clientSecret: "test-client-secret",
},
},
oauthClientMock,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider refactoring repeated ServerClient initialization

The initialization of ServerClient with mocked OAuth tokens in this test is repeated in other tests (e.g., lines 626-638 and 673-707). To improve maintainability and reduce code duplication, consider refactoring this setup into a shared helper function or utilizing a common beforeEach hook where appropriate.

Comment on lines 832 to 858
public async invokeWorkflowForExternalUser(url: string, externalUserId: string, opts: RequestOptions = {}): Promise<unknown> {
const {
body,
headers = {},
} = opts;

if (!externalUserId) {
throw new Error("External user ID is required");
}

if (!this.publicKey) {
throw new Error("Project public key is required to map the external user ID to the correct project");
}

return this.makeRequest("", {
...opts,
baseURL: url,
method: opts.method || "POST", // Default to POST if not specified
headers: {
...headers,
"Authorization": await this.oauthAuthorizationHeader(),
"X-PD-External-User-ID": externalUserId,
"X-PD-Project-Public-Key": this.publicKey,
},
body,
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add input validation for the url parameter

In the invokeWorkflowForExternalUser method, the url parameter is used without validation. To prevent runtime errors and enhance security, consider validating the url to ensure it is a well-formed URL.

Add URL validation:

 public async invokeWorkflowForExternalUser(
   url: string,
   externalUserId: string,
   opts: RequestOptions = {},
 ): Promise<unknown> {
+  try {
+    new URL(url);
+  } catch {
+    throw new Error(`Invalid URL provided: ${url}`);
+  }

   const {
     body,
     headers = {},
   } = opts;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public async invokeWorkflowForExternalUser(url: string, externalUserId: string, opts: RequestOptions = {}): Promise<unknown> {
const {
body,
headers = {},
} = opts;
if (!externalUserId) {
throw new Error("External user ID is required");
}
if (!this.publicKey) {
throw new Error("Project public key is required to map the external user ID to the correct project");
}
return this.makeRequest("", {
...opts,
baseURL: url,
method: opts.method || "POST", // Default to POST if not specified
headers: {
...headers,
"Authorization": await this.oauthAuthorizationHeader(),
"X-PD-External-User-ID": externalUserId,
"X-PD-Project-Public-Key": this.publicKey,
},
body,
});
}
public async invokeWorkflowForExternalUser(url: string, externalUserId: string, opts: RequestOptions = {}): Promise<unknown> {
try {
new URL(url);
} catch {
throw new Error(`Invalid URL provided: ${url}`);
}
const {
body,
headers = {},
} = opts;
if (!externalUserId) {
throw new Error("External user ID is required");
}
if (!this.publicKey) {
throw new Error("Project public key is required to map the external user ID to the correct project");
}
return this.makeRequest("", {
...opts,
baseURL: url,
method: opts.method || "POST", // Default to POST if not specified
headers: {
...headers,
"Authorization": await this.oauthAuthorizationHeader(),
"X-PD-External-User-ID": externalUserId,
"X-PD-Project-Public-Key": this.publicKey,
},
body,
});
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (1)
packages/sdk/src/server/index.ts (1)

798-851: New invokeWorkflowForExternalUser method enhances API functionality

The new method provides a valuable addition to the API, allowing invocation of workflows for specific external users. The implementation is well-structured and includes proper error checking.

Suggestion for improvement:
Consider adding input validation for the url parameter to ensure it's a valid URL. This can help prevent runtime errors and enhance security.

Example implementation:

public async invokeWorkflowForExternalUser(
  url: string,
  externalUserId: string,
  opts: RequestOptions = {}
): Promise<unknown> {
  try {
    new URL(url);
  } catch {
    throw new Error(`Invalid URL provided: ${url}`);
  }

  // ... rest of the method implementation
}
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 6d24e66 and 651d41b.

📒 Files selected for processing (1)
  • packages/sdk/src/server/index.ts (4 hunks)
🔇 Additional comments not posted (2)
packages/sdk/src/server/index.ts (2)

10-18: New type definitions enhance code structure and type safety

The introduction of ProjectKeys and PipedreamOAuthClient types improves code organization and provides better type safety for project-related and OAuth-related data structures.


26-57: Improved structure and deprecation notices in CreateServerClientOpts

The changes to CreateServerClientOpts enhance the configuration structure by introducing project and oauth properties, while properly deprecating the older properties. This approach facilitates a smooth transition to the new structure while maintaining backward compatibility.

Comment on lines 337 to 340
if (opts.project) {
this.publicKey = opts.project.publicKey;
this.secretKey = opts.project.secretKey;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Consider clarifying precedence of new vs. deprecated properties

The current implementation overwrites the deprecated properties with the new ones from the project object. This behavior might lead to unexpected results if both sets of properties are provided.

Consider the following improvements:

  1. Add a warning log when both sets of properties are provided.
  2. Implement a clear precedence rule, e.g., prefer the new project object over deprecated properties.
  3. Document this behavior in the class or method documentation.

Example implementation:

if (opts.project) {
  if (this.publicKey || this.secretKey) {
    console.warn('Both deprecated and new project properties provided. Using new project properties.');
  }
  this.publicKey = opts.project.publicKey;
  this.secretKey = opts.project.secretKey;
}

Comment on lines 362 to 370
if (!oauth || !id || !secret) {
return;
}

const client = {
id: id ?? oauth.clientId,
secret: secret ?? oauth.clientSecret,
};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix logical error in OAuth client configuration

The current implementation has a logical error that prevents the OAuth client from being configured when only the new oauth object is provided. To address this issue and ensure proper configuration, please apply the following changes:

-if (!oauth || !id || !secret) {
+if (!(oauth || (id && secret))) {
  return;
}

const client = {
-  id: id ?? oauth.clientId,
-  secret: secret ?? oauth.clientSecret,
+  id: oauth?.clientId ?? id,
+  secret: oauth?.clientSecret ?? secret,
};

This change ensures that the OAuth client is configured correctly when either the new oauth object or the deprecated properties are provided.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (!oauth || !id || !secret) {
return;
}
const client = {
id: id ?? oauth.clientId,
secret: secret ?? oauth.clientSecret,
};
if (!(oauth || (id && secret))) {
return;
}
const client = {
id: oauth?.clientId ?? id,
secret: oauth?.clientSecret ?? secret,
};

Correcting external user ID method.

Correcting API request base method to handle authType
@dylburger dylburger changed the title Adding new project? and oauth? params to CreateServerClientOpts, invokeWorkflowForExternalUser method Fixes DJ-2470 Oct 7, 2024
Copy link

linear bot commented Oct 7, 2024


/**
* A unique identifier for an app.
* The name slug for an app, a unique, human-readable identifier like "github" or "google_sheets".
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


describe("ServerClient", () => {
let client: ServerClient;
let client: ServerClient;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this up to the beforeEach block. Previously I'd been duplicating the client in many tests (see removal below)

const getTokenMock = jest.fn().mockResolvedValue({
token: {
access_token: "mocked-oauth-token",
it("should invoke a workflow with provided URL and body, with no auth type", async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy path — default invokeWorkflow has an HTTPAuthType of None


// Inject the mock oauthClient into the ServerClient instance
client = new ServerClient(
it("should invoke a workflow with OAuth auth type", async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New test for OAuth

);
});

it("should invoke a workflow with static bearer auth type", async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New test for static bearer token (the expectation is that the customer passes this in themselves in the Authorization header, as we'll document

});
});

describe("invokeWorkflowForExternalUser", () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New tests for invokeWorkflowForExternalUser

bzwrk
bzwrk previously approved these changes Oct 28, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Outside diff range and nitpick comments (2)
.github/workflows/pipedream-sdk-markdown-lint.yaml (1)

17-20: Consider adding configuration file for markdownlint.

While the current setup works, consider adding a .markdownlint.json or .markdownlint.yaml configuration file to customize the linting rules specific to your project's needs.

Would you like me to help generate a base configuration file with recommended rules for your SDK documentation?

packages/sdk/src/server/index.ts (1)

751-846: Consider implementing rate limiting for workflow invocations.

The workflow invocation methods could benefit from rate limiting to prevent abuse and ensure fair usage of resources.

Consider implementing a rate limiter using a token bucket algorithm:

interface RateLimitOptions {
  maxRequests: number;
  timeWindow: number; // in milliseconds
}

class RateLimiter {
  private tokens: number;
  private lastRefill: number;
  
  constructor(private options: RateLimitOptions) {
    this.tokens = options.maxRequests;
    this.lastRefill = Date.now();
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens <= 0) {
      throw new Error('Rate limit exceeded');
    }
    this.tokens--;
  }

  private refill(): void {
    const now = Date.now();
    const timePassed = now - this.lastRefill;
    const refillAmount = Math.floor(timePassed / this.options.timeWindow * this.options.maxRequests);
    this.tokens = Math.min(this.options.maxRequests, this.tokens + refillAmount);
    this.lastRefill = now;
  }
}

This can be integrated into the workflow invocation methods to enforce rate limits.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between d27a9b1 and a97736d.

📒 Files selected for processing (2)
  • .github/workflows/pipedream-sdk-markdown-lint.yaml (1 hunks)
  • packages/sdk/src/server/index.ts (18 hunks)
🔇 Additional comments (1)
packages/sdk/src/server/index.ts (1)

10-60: LGTM! Well-structured type definitions and enums.

The new types and enums are well-documented and follow TypeScript best practices. The HTTPAuthType enum values are consistent with the backend implementation.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

🧹 Outside diff range and nitpick comments (6)
.github/workflows/pipedream-sdk-markdown-lint.yaml (1)

21-24: Consider adding a markdownlint configuration file.

While the linting action is correctly configured, consider adding a .markdownlint.json or .markdownlint.yaml configuration file to ensure consistent linting rules across the SDK documentation.

Example configuration file:

{
  "default": true,
  "MD013": false,  # Line length
  "MD033": false,  # Inline HTML
  "MD041": false   # First line in file should be a top level heading
}
packages/sdk/CONTRIBUTING.md (3)

16-19: Add comma for better readability

Add a comma after "tool" to improve sentence structure.

-If you prefer to use another tool make sure you use the same versions that are
+If you prefer to use another tool, make sure you use the same versions that are
🧰 Tools
🪛 LanguageTool

[uncategorized] ~16-~16: A comma might be missing here.
Context: ...stall ``` If you prefer to use another tool make sure you use the same versions tha...

(AI_EN_LECTOR_MISSING_PUNCTUATION_COMMA)


23-31: Consider mentioning root dependencies

The instructions should mention that there are dependencies in the root package.json that might need to be installed first. This ensures a complete setup process.

Consider adding:

 Since other packages in this repository already use `pnpm`, we recommend you use
 it in this case too to keep your `node_modules` footprint low.
+
+Note: You may also need to install dependencies from the root `package.json`. Run
+`pnpm install` from the root directory first.

34-45: Add context about build artifacts

Consider adding information about:

  1. What artifacts are generated
  2. Where to find them
  3. Common troubleshooting steps
 There's a script that you can invoke with `pnpm` to build the package artifacts:

 ```shell
 pnpm build

+This will generate the compiled artifacts in the dist/ directory. If you
+encounter any build errors, ensure all dependencies are installed correctly.

You can also watch the code for changes, and automatically build a new artifact
after each change with the watch script:


</blockquote></details>
<details>
<summary>packages/sdk/src/server/__tests__/server.test.ts (2)</summary><blockquote>

`165-184`: **Consider testing additional OAuth error scenarios.**

The OAuth token retrieval failure test only covers the "Invalid credentials" case. Consider adding tests for other common OAuth failure scenarios like network errors or token endpoint failures.

---

`727-733`: **Consider additional validation test cases.**

While testing for empty `externalUserId` is good, consider adding tests for:
- null/undefined `externalUserId`
- whitespace-only `externalUserId`
- extremely long `externalUserId` values

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between a97736d1c7a01c31afef0ff9ffab5debba5273b7 and 6596349521e92baac1341b84a950835e0a1033f5.

</details>

<details>
<summary>⛔ Files ignored due to path filters (1)</summary>

* `pnpm-lock.yaml` is excluded by `!**/pnpm-lock.yaml`

</details>

<details>
<summary>📒 Files selected for processing (6)</summary>

* `.github/workflows/pipedream-sdk-markdown-lint.yaml` (1 hunks)
* `.github/workflows/pipedream-sdk-test.yaml` (1 hunks)
* `packages/sdk/CONTRIBUTING.md` (1 hunks)
* `packages/sdk/src/server/__tests__/server.test.ts` (23 hunks)
* `packages/sdk/src/server/index.ts` (18 hunks)
* `packages/sdk/tsconfig.node.json` (1 hunks)

</details>

<details>
<summary>🚧 Files skipped from review as they are similar to previous changes (1)</summary>

* packages/sdk/tsconfig.node.json

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>packages/sdk/CONTRIBUTING.md</summary>

[uncategorized] ~16-~16: A comma might be missing here.
Context: ...stall ```  If you prefer to use another tool make sure you use the same versions tha...

(AI_EN_LECTOR_MISSING_PUNCTUATION_COMMA)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (6)</summary>

<details>
<summary>.github/workflows/pipedream-sdk-markdown-lint.yaml (3)</summary>

`3-8`: **LGTM! Well-configured triggers and path filters.**

The workflow is correctly configured to run only when relevant Markdown files in the SDK package are modified.

---

`9-11`: **LGTM! Effective concurrency configuration.**

The concurrency settings will properly manage simultaneous workflow runs, canceling outdated ones.

---

`6-7`: **Verify the existence of SDK documentation files.**

Let's ensure there are markdown files to lint in the specified path.

<details>
<summary>✅ Verification successful</summary>

**Markdown files exist in the SDK package - workflow path pattern is valid**

The verification confirms the existence of multiple markdown files (`README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`) in the `packages/sdk/` directory, which matches the workflow's path pattern `'packages/sdk/**/*.md'`. The workflow will correctly lint these documentation files.



</details>

<details>
<summary>🏁 Scripts executed</summary>

The following scripts were executed for the analysis:

Script:

```shell
#!/bin/bash
# Description: Verify the existence of markdown files in the SDK package
# Expected: List of markdown files that will be linted by this workflow

echo "Checking for markdown files in SDK package..."
fd -e md . packages/sdk/

Length of output: 205

.github/workflows/pipedream-sdk-test.yaml (1)

9-11: LGTM! Good addition of concurrency controls.

The concurrency configuration is well-implemented, helping to optimize CI resources by preventing redundant runs and automatically canceling outdated ones when new commits are pushed.

packages/sdk/src/server/__tests__/server.test.ts (2)

685-692: Duplicate client initialization detected.

Similar to the previous comment, this beforeEach block also duplicates client initialization logic.


736-816: LGTM: Comprehensive URL building test coverage.

The URL building tests are well-structured and cover various scenarios including:

  • Input validation
  • Protocol handling
  • Path handling
  • Custom domain support

steps:
- name: Checkout code
uses: actions/checkout@v3
uses: actions/checkout@v4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Update actions/checkout to v4.1.7 for version consistency

Most workflows in the repository are using actions/checkout@v4.1.7, while this workflow and pipedream-sdk-markdown-lint.yaml are using the less specific actions/checkout@v4. For consistency across the codebase, update the version to v4.1.7.

  • .github/workflows/pipedream-sdk-test.yaml: Update actions/checkout@v4 to actions/checkout@v4.1.7
  • .github/workflows/pipedream-sdk-markdown-lint.yaml: Update actions/checkout@v4 to actions/checkout@v4.1.7
🔗 Analysis chain

Verify compatibility with actions/checkout@v4.

The upgrade to actions/checkout@v4 is a good practice for keeping dependencies current. However, as this is a major version update, please ensure all workflow features remain compatible.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any other workflows using actions/checkout to ensure consistent versioning
# Test: Search for checkout action usage. Expect: All workflows should use v4 for consistency

echo "Checking for inconsistent versions of actions/checkout across workflows..."
rg -g "*.ya?ml" "uses: actions/checkout@" .github/workflows/

Length of output: 221


Script:

#!/bin/bash
# Let's try with a different approach using fd to find yaml files first
echo "GitHub workflow files in the repository:"
fd -e yml -e yaml . .github/workflows/

echo -e "\nChecking checkout action usage in these files:"
fd -e yml -e yaml . .github/workflows/ -x grep -l "actions/checkout@" {} \;

echo -e "\nDetailed checkout action versions in use:"
fd -e yml -e yaml . .github/workflows/ -x grep -H "actions/checkout@" {} \;

Length of output: 2284

Comment on lines +70 to +73
# Replace /path/to/pipedream with the actual path to this repository
grep nodejs /path/to/pipedream/packages/sdk/.tool-versions >> .tool-versions
asdf install
pnpm install @pipedream/sdk
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider automating Node.js version detection

The manual path replacement and version copying could be error-prone. Consider providing a script to automate this process.

-# Replace /path/to/pipedream with the actual path to this repository
-grep nodejs /path/to/pipedream/packages/sdk/.tool-versions >> .tool-versions
-asdf install
+# Create a setup script (e.g., setup.sh):
+#!/bin/bash
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+SDK_DIR=$(find "$SCRIPT_DIR" -name "packages/sdk" -type d | head -n 1)
+if [ -n "$SDK_DIR" ]; then
+  grep nodejs "$SDK_DIR/.tool-versions" >> .tool-versions
+  asdf install
+else
+  echo "Error: Could not locate SDK directory"
+  exit 1
+fi

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines 56 to 60
export enum HTTPAuthType {
None = "none",
StaticBearer = "static_bearer_token",
OAuth = "oauth"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider using const enum for better type safety.

The HTTPAuthType enum could benefit from being a const enum to ensure values are inlined at compile time and prevent runtime manipulation.

-export enum HTTPAuthType {
+export const enum HTTPAuthType {
   None = "none",
   StaticBearer = "static_bearer_token",
   OAuth = "oauth"
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export enum HTTPAuthType {
None = "none",
StaticBearer = "static_bearer_token",
OAuth = "oauth"
}
export const enum HTTPAuthType {
None = "none",
StaticBearer = "static_bearer_token",
OAuth = "oauth"
}

Comment on lines 848 to 869
public async invokeWorkflowForExternalUser(
url: string,
externalUserId: string,
opts: RequestOptions = {},
): Promise<unknown> {
const { headers = {} } = opts;

if (!externalUserId) {
throw new Error("External user ID is required");
}

if (!this.oauthClient) {
throw new Error("OAuth is required for invoking workflows for external users. Please pass credentials for a valid OAuth client");
}

return this.invokeWorkflow(url, {
...opts,
headers: {
...headers,
"Authorization": await this.oauthAuthorizationHeader(),
"X-PD-External-User-ID": externalUserId,
},
body,
});
}, HTTPAuthType.OAuth); // OAuth auth is required for invoking workflows for external users
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Enhance error handling in workflow invocation.

The invokeWorkflowForExternalUser method could benefit from additional validation and error handling.

 public async invokeWorkflowForExternalUser(
   url: string,
   externalUserId: string,
   opts: RequestOptions = {},
 ): Promise<unknown> {
   const { headers = {} } = opts;

-  if (!externalUserId) {
+  if (!externalUserId?.trim()) {
     throw new Error("External user ID is required");
   }

+  if (!url?.trim()) {
+    throw new Error("Workflow URL is required");
+  }

   if (!this.oauthClient) {
     throw new Error("OAuth is required for invoking workflows for external users. Please pass credentials for a valid OAuth client");
   }

+  try {
     return this.invokeWorkflow(url, {
       ...opts,
       headers: {
         ...headers,
         "X-PD-External-User-ID": externalUserId,
       },
     }, HTTPAuthType.OAuth);
+  } catch (error) {
+    throw new Error(`Failed to invoke workflow for external user: ${error instanceof Error ? error.message : String(error)}`);
+  }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public async invokeWorkflowForExternalUser(
url: string,
externalUserId: string,
opts: RequestOptions = {},
): Promise<unknown> {
const { headers = {} } = opts;
if (!externalUserId) {
throw new Error("External user ID is required");
}
if (!this.oauthClient) {
throw new Error("OAuth is required for invoking workflows for external users. Please pass credentials for a valid OAuth client");
}
return this.invokeWorkflow(url, {
...opts,
headers: {
...headers,
"Authorization": await this.oauthAuthorizationHeader(),
"X-PD-External-User-ID": externalUserId,
},
body,
});
}, HTTPAuthType.OAuth); // OAuth auth is required for invoking workflows for external users
public async invokeWorkflowForExternalUser(
url: string,
externalUserId: string,
opts: RequestOptions = {},
): Promise<unknown> {
const { headers = {} } = opts;
if (!externalUserId?.trim()) {
throw new Error("External user ID is required");
}
if (!url?.trim()) {
throw new Error("Workflow URL is required");
}
if (!this.oauthClient) {
throw new Error("OAuth is required for invoking workflows for external users. Please pass credentials for a valid OAuth client");
}
try {
return this.invokeWorkflow(url, {
...opts,
headers: {
...headers,
"X-PD-External-User-ID": externalUserId,
},
}, HTTPAuthType.OAuth); // OAuth auth is required for invoking workflows for external users
} catch (error) {
throw new Error(`Failed to invoke workflow for external user: ${error instanceof Error ? error.message : String(error)}`);
}
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (5)
packages/sdk/src/server/index.ts (1)

352-373: Enhance OAuth error handling with specific error types.

The OAuth client initialization could benefit from more specific error handling to help users troubleshoot common OAuth issues.

Apply this change:

 private newOauthClient(
   { clientId, clientSecret }: OAuthCredentials,
   tokenHost: string,
 ) {
   if (!clientId || !clientSecret) {
-    throw new Error("OAuth client ID and secret are required");
+    throw new Error("OAuth configuration error: Both client ID and secret are required");
   }

   const client = {
     id: clientId,
     secret: clientSecret,
   };
-  return new ClientCredentials({
-    client,
-    auth: {
-      tokenHost,
-      tokenPath: "/v1/oauth/token",
-    },
-  });
+  try {
+    return new ClientCredentials({
+      client,
+      auth: {
+        tokenHost,
+        tokenPath: "/v1/oauth/token",
+      },
+    });
+  } catch (error) {
+    throw new Error(`OAuth initialization failed: ${error instanceof Error ? error.message : 'Unknown error'}`);
+  }
packages/sdk/src/server/__tests__/server.test.ts (4)

23-41: Consider extracting client initialization into a helper function.

The client initialization logic is duplicated across multiple test blocks. Despite previous suggestions, this pattern continues throughout the file.

function createTestClient(options: Partial<BackendClientOpts> = {}) {
  return new BackendClient({
    ...clientParams,
    ...options,
  });
}

beforeEach(() => {
  setupOAuthMocks();
  client = createTestClient();
  customDomainClient = createTestClient({ workflowDomain: "example.com" });
});

Line range hint 616-682: Add comments explaining the OAuth token lifecycle test.

While the test is well-structured, adding comments to explain the token lifecycle stages (initial token acquisition, expiration check, refresh) would improve maintainability.

 describe("OAuth Token Handling", () => {
   it("should refresh token when expired", async () => {
+    // Stage 1: Setup initial expired token
     const expiredTokenMock = {
       token: {
         access_token: "expired-oauth-token",
       },
       expired: jest.fn().mockReturnValue(true),
     };

+    // Stage 2: Setup refresh token response
     const newTokenMock = {
       token: {
         access_token: "new-oauth-token",
       },
       expired: jest.fn().mockReturnValue(false),
     };

+    // Stage 3: Configure token lifecycle (expired -> refreshed)
     const getTokenMock = jest
       .fn()
       .mockResolvedValueOnce(expiredTokenMock)
       .mockResolvedValueOnce(newTokenMock);

684-750: Consider adding environment-specific test cases.

The tests validate the presence of environment headers but don't verify behavior across different environments (e.g., development, staging).

Add test cases for different environments:

it("should support different environments", async () => {
  const environments = ["development", "staging", "production"];
  for (const env of environments) {
    const client = new BackendClient({
      ...clientParams,
      workflowDomain: "example.com",
      environment: env,
    });
    
    // Test implementation
    const result = await client.invokeWorkflowForExternalUser(
      "https://example.com/workflow",
      "external-user-id",
      { body: { foo: "bar" } }
    );
    
    expect(fetchMock).toHaveBeenCalledWith(
      expect.any(String),
      expect.objectContaining({
        headers: expect.objectContaining({
          "X-PD-Environment": env,
        }),
      })
    );
  }
});

752-832: Add test cases for URL edge cases.

While the current test coverage is good, consider adding tests for:

  • URLs with query parameters
  • URLs with fragments
  • URLs with special characters in the path
describe("Edge cases", () => {
  it("should handle URLs with query parameters", () => {
    const input = "https://en123.example.com/foo?key=value";
    const expected = "https://en123.example.com/foo?key=value";
    expect(customDomainClient["buildWorkflowUrl"](input)).toBe(expected);
  });

  it("should handle URLs with fragments", () => {
    const input = "https://en123.example.com/foo#section";
    const expected = "https://en123.example.com/foo#section";
    expect(customDomainClient["buildWorkflowUrl"](input)).toBe(expected);
  });

  it("should handle URLs with encoded characters", () => {
    const input = "https://en123.example.com/foo%20bar";
    const expected = "https://en123.example.com/foo%20bar";
    expect(customDomainClient["buildWorkflowUrl"](input)).toBe(expected);
  });
});
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 6596349 and 401522c.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (3)
  • packages/sdk/package.json (3 hunks)
  • packages/sdk/src/server/__tests__/server.test.ts (23 hunks)
  • packages/sdk/src/server/index.ts (18 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/sdk/package.json
🧰 Additional context used
🪛 Biome
packages/sdk/src/server/index.ts

[error] 56-60: The enum declaration should not be const

Const enums are not supported by bundlers and are incompatible with the 'isolatedModules' mode. Their use can lead to import inexistent values.
See TypeScript Docs for more details.
Safe fix: Turn the const enum into a regular enum.

(lint/suspicious/noConstEnum)


[error] 140-144: The enum declaration should not be const

Const enums are not supported by bundlers and are incompatible with the 'isolatedModules' mode. Their use can lead to import inexistent values.
See TypeScript Docs for more details.
Safe fix: Turn the const enum into a regular enum.

(lint/suspicious/noConstEnum)

🔇 Additional comments (3)
packages/sdk/src/server/index.ts (3)

665-741: LGTM! Robust URL validation implementation.

The URL building logic includes:

  • Proper input sanitization
  • Protection against DNS rebinding attacks
  • Comprehensive validation of endpoint ID format
  • Clear error messages

775-813: LGTM! Well-implemented workflow invocation methods.

The implementation includes:

  • Proper input validation
  • Clear auth type handling
  • Comprehensive error messages
  • Proper external user ID validation

Also applies to: 848-872


334-349: 🛠️ Refactor suggestion

Add validation for environment parameter.

The constructor accepts any string for the environment parameter. Consider restricting it to specific valid environments to prevent configuration errors.

Apply this change:

 constructor(opts: BackendClientOpts) {
-  this.environment = opts.environment ?? "production";
+  const validEnvironments = ["production", "development"] as const;
+  const environment = opts.environment ?? "production";
+  if (environment && !validEnvironments.includes(environment as typeof validEnvironments[number])) {
+    throw new Error(`Invalid environment. Must be one of: ${validEnvironments.join(", ")}`);
+  }
+  this.environment = environment;

Likely invalid or redundant comment.

Comment on lines +56 to +60
export const enum HTTPAuthType {
None = "none",
StaticBearer = "static_bearer_token",
OAuth = "oauth"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Convert const enums to regular enums to ensure compatibility.

The const enum usage in HTTPAuthType and AppAuthType can cause compatibility issues with bundlers and TypeScript's isolatedModules mode.

Apply these changes:

-export const enum HTTPAuthType {
+export enum HTTPAuthType {
   None = "none",
   StaticBearer = "static_bearer_token",
   OAuth = "oauth"
 }

-export const enum AppAuthType {
+export enum AppAuthType {
   OAuth = "oauth",
   Keys = "keys",
   None = "none",
}

Also applies to: 140-144

🧰 Tools
🪛 Biome

[error] 56-60: The enum declaration should not be const

Const enums are not supported by bundlers and are incompatible with the 'isolatedModules' mode. Their use can lead to import inexistent values.
See TypeScript Docs for more details.
Safe fix: Turn the const enum into a regular enum.

(lint/suspicious/noConstEnum)

@dylburger dylburger merged commit 91ea950 into master Nov 4, 2024
14 checks passed
@dylburger dylburger deleted the sdk/connect-workflow-invocation branch November 4, 2024 21:15
lcaresia pushed a commit that referenced this pull request Dec 3, 2024
@pipedream/sdk 1.0.0

---------

Co-authored-by: Jay Vercellone <jay@pipedream.com>
Co-authored-by: Jay Vercellone <jverce@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@coderabbitai coderabbitai bot mentioned this pull request Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants