Skip to content

Waitpoint token callback URLs #2025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
May 7, 2025
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
62135c6
Initial commit with a plan for what we’re going to do
matt-aitken May 2, 2025
78f534e
Some initial types and improved plan
matt-aitken May 2, 2025
d9aea07
Add Waitpoint resolver
matt-aitken May 5, 2025
87e107d
Add resolver + status index
matt-aitken May 5, 2025
59b13f6
Remove type + status index
matt-aitken May 5, 2025
30a5497
Only drop if exists
matt-aitken May 5, 2025
b7218af
Remove type index
matt-aitken May 5, 2025
601002b
Update waitpoint list presenter to use resolver
matt-aitken May 5, 2025
79e98c1
Added resolver to the engine
matt-aitken May 5, 2025
1c09fbe
Made the existing waitpoint list presenter more flexible
matt-aitken May 5, 2025
b9edd2e
Initial implentation ofr wait.forHttpCallback()
matt-aitken May 5, 2025
93fbfb4
Added the callback endpoint (no API rate limit)
matt-aitken May 5, 2025
6df4084
schema version
matt-aitken May 5, 2025
676f676
Added jsdocs, removed schema version because of errors
matt-aitken May 5, 2025
27f2bde
Show callback URL if it’s set
matt-aitken May 5, 2025
25448bd
Dashboard pages and panels
matt-aitken May 5, 2025
8bfc515
Remove todos
matt-aitken May 5, 2025
2b21ca3
Added temporary icon
matt-aitken May 5, 2025
f5d73f4
Added a blank state
matt-aitken May 5, 2025
1e5350a
Some tweaks and added a Replicate example
matt-aitken May 6, 2025
71d88b9
Implement unwrap() for httpCallback
matt-aitken May 6, 2025
7766c2f
Added unwrap to wait.forToken() as well
matt-aitken May 6, 2025
54e76ce
Improved jsdocs
matt-aitken May 6, 2025
6cd8870
Added docs
matt-aitken May 6, 2025
3f50577
Added unwrap to the token docs
matt-aitken May 6, 2025
c48b27a
Merge branch 'main' into wait-for-http-callback
matt-aitken May 6, 2025
455f5a2
Show a dash if there are no tags
matt-aitken May 6, 2025
d1c19fc
Make the timeout error safer
matt-aitken May 6, 2025
e2eb321
Fixed migrations… should use id desc not createdAt desc
matt-aitken May 6, 2025
b218ff8
Fixed page title
matt-aitken May 6, 2025
8503cea
Fixed migration so it only adds them if they don’t exist. This allows…
matt-aitken May 6, 2025
799f8e1
Respect the max content length by getting the length of the body
matt-aitken May 7, 2025
ec9fdf6
Added more docs details about the callback format
matt-aitken May 7, 2025
49e7f15
Remove code comment
matt-aitken May 7, 2025
aad1278
Improved the error
matt-aitken May 7, 2025
17bcce6
Added a hash to the HTTP callback URLs
matt-aitken May 7, 2025
46c34f3
Add the apiKey to the API input type to fix TS error
matt-aitken May 7, 2025
344dfde
Return the error responses. They were being caught and not preserved
matt-aitken May 7, 2025
fcbc275
The content-length header is required. Deal with an empty body
matt-aitken May 7, 2025
4b790bd
Removed unused types
matt-aitken May 7, 2025
c0d01e4
Added some new span icons
matt-aitken May 7, 2025
beca6ea
Reworked http callback to be a create call then just use wait.forToken()
matt-aitken May 7, 2025
4e9c2ba
Added a changeset
matt-aitken May 7, 2025
b87d98f
Updated the docs
matt-aitken May 7, 2025
2536c10
Updated the wait overview docs
matt-aitken May 7, 2025
d8c76ee
Simplify to just a call
matt-aitken May 7, 2025
9d01c88
WIP stripping right back to waitpoints just having a URL associated w…
matt-aitken May 7, 2025
93263d9
More deletions
matt-aitken May 7, 2025
f7af73f
Remove missing icon
matt-aitken May 7, 2025
2710501
Updated the changeset
matt-aitken May 7, 2025
4b24a47
Add URL to the token return types
matt-aitken May 7, 2025
776b7ea
Remove wait for http callback page
matt-aitken May 7, 2025
474a333
Updated docs
matt-aitken May 7, 2025
e2b39cb
More tidying
matt-aitken May 7, 2025
a725140
Type and import fix
matt-aitken May 7, 2025
0a97f3a
Remove unused import
matt-aitken May 7, 2025
d4b7135
Some type fixes for the retrieve
matt-aitken May 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions apps/webapp/app/assets/icons/HttpCallbackIcon.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
export function HttpCallbackIcon({ className }: { className?: string }) {
return (
<svg
className={className}
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
strokeLinecap="round"
strokeLinejoin="round"
>
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
<path d="M21 12a9 9 0 1 0 -9 9" />
<path d="M3.6 9h16.8" />
<path d="M3.6 15h8.4" />
<path d="M11.578 3a17 17 0 0 0 0 18" />
<path d="M12.5 3c1.719 2.755 2.5 5.876 2.5 9" />
<path d="M18 21v-7m3 3l-3 -3l-3 3" />
</svg>
);
}
28 changes: 28 additions & 0 deletions apps/webapp/app/components/BlankStatePanels.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ import { TextLink } from "./primitives/TextLink";
import { InitCommandV3, PackageManagerProvider, TriggerDevStepV3 } from "./SetupCommands";
import { StepContentContainer } from "./StepContentContainer";
import { WaitpointTokenIcon } from "~/assets/icons/WaitpointTokenIcon";
import { HttpCallbackIcon } from "~/assets/icons/HttpCallbackIcon";

export function HasNoTasksDev() {
return (
Expand Down Expand Up @@ -431,6 +432,33 @@ export function NoWaitpointTokens() {
);
}

export function NoHttpCallbacks() {
return (
<InfoPanel
title="You don't have any HTTP callbacks"
icon={HttpCallbackIcon}
iconClassName="text-teal-500"
panelClassName="max-w-md"
accessory={
<LinkButton
to={docsPath("wait-for-http-callback")}
variant="docs/small"
LeadingIcon={BookOpenIcon}
>
Waitpoint docs
</LinkButton>
}
>
<Paragraph spacing variant="small">
HTTP callbacks are used to pause runs until an HTTP request is made to a provided URL.
</Paragraph>
<Paragraph spacing variant="small">
They are useful when using APIs that provide a callback URL. You can send the URL to them
and when they callback your run will continue.
</Paragraph>
</InfoPanel>
);
}
function SwitcherPanel() {
const organization = useOrganization();
const project = useProject();
Expand Down
8 changes: 8 additions & 0 deletions apps/webapp/app/components/navigation/SideMenu.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import {
v3SchedulesPath,
v3TestPath,
v3UsagePath,
v3WaitpointHttpCallbacksPath,
v3WaitpointTokensPath,
} from "~/utils/pathBuilder";
import { useKapaWidget } from "../../hooks/useKapaWidget";
Expand All @@ -80,6 +81,7 @@ import { HelpAndFeedback } from "./HelpAndFeedbackPopover";
import { SideMenuHeader } from "./SideMenuHeader";
import { SideMenuItem } from "./SideMenuItem";
import { SideMenuSection } from "./SideMenuSection";
import { HttpCallbackIcon } from "~/assets/icons/HttpCallbackIcon";

type SideMenuUser = Pick<User, "email" | "admin"> & { isImpersonating: boolean };
export type SideMenuProject = Pick<
Expand Down Expand Up @@ -245,6 +247,12 @@ export function SideMenu({
activeIconColor="text-sky-500"
to={v3WaitpointTokensPath(organization, project, environment)}
/>
<SideMenuItem
name="HTTP callbacks"
icon={HttpCallbackIcon}
activeIconColor="text-teal-500"
to={v3WaitpointHttpCallbacksPath(organization, project, environment)}
/>
</SideMenuSection>

<SideMenuSection title="Manage">
Expand Down
56 changes: 52 additions & 4 deletions apps/webapp/app/components/runs/v3/WaitpointDetails.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,19 @@ import { useOrganization } from "~/hooks/useOrganizations";
import { useProject } from "~/hooks/useProject";
import { type WaitpointDetail } from "~/presenters/v3/WaitpointPresenter.server";
import { ForceTimeout } from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.waitpoints.$waitpointFriendlyId.complete/route";
import { v3WaitpointTokenPath, v3WaitpointTokensPath } from "~/utils/pathBuilder";
import {
v3WaitpointHttpCallbackPath,
v3WaitpointTokenPath,
v3WaitpointTokensPath,
} from "~/utils/pathBuilder";
import { PacketDisplay } from "./PacketDisplay";
import { WaitpointStatusCombo } from "./WaitpointStatus";
import { RunTag } from "./RunTag";
import { CopyableText } from "~/components/primitives/CopyableText";
import { ClipboardField } from "~/components/primitives/ClipboardField";
import { WaitpointResolver } from "@trigger.dev/database";
import { WaitpointTokenIcon } from "~/assets/icons/WaitpointTokenIcon";
import { HttpCallbackIcon } from "~/assets/icons/HttpCallbackIcon";

export function WaitpointDetailTable({
waitpoint,
Expand Down Expand Up @@ -39,9 +48,15 @@ export function WaitpointDetailTable({
<Property.Value className="whitespace-pre-wrap">
{linkToList ? (
<TextLink
to={v3WaitpointTokenPath(organization, project, environment, waitpoint, {
id: waitpoint.id,
})}
to={
waitpoint.resolver === "TOKEN"
? v3WaitpointTokenPath(organization, project, environment, waitpoint, {
id: waitpoint.id,
})
: v3WaitpointHttpCallbackPath(organization, project, environment, waitpoint, {
id: waitpoint.id,
})
}
>
{waitpoint.id}
</TextLink>
Expand All @@ -50,6 +65,20 @@ export function WaitpointDetailTable({
)}
</Property.Value>
</Property.Item>
<Property.Item>
<Property.Label>Type</Property.Label>
<Property.Value>
<WaitpointResolverCombo resolver={waitpoint.resolver} />
</Property.Value>
</Property.Item>
{waitpoint.callbackUrl && (
<Property.Item>
<Property.Label>Callback URL</Property.Label>
<Property.Value className="my-1">
<ClipboardField value={waitpoint.callbackUrl} variant={"secondary/small"} />
</Property.Value>
</Property.Item>
)}
<Property.Item>
<Property.Label>Idempotency key</Property.Label>
<Property.Value>
Expand Down Expand Up @@ -128,3 +157,22 @@ export function WaitpointDetailTable({
</Property.Table>
);
}

export function WaitpointResolverCombo({ resolver }: { resolver: WaitpointResolver }) {
switch (resolver) {
case "TOKEN":
return (
<div className="flex items-center gap-1">
<WaitpointTokenIcon className="size-4" />
<span>Token</span>
</div>
);
case "HTTP_CALLBACK":
return (
<div className="flex items-center gap-1">
<HttpCallbackIcon className="size-4" />
<span>HTTP Callback</span>
</div>
);
}
}
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
import { RuntimeEnvironmentType, WaitpointTokenStatus } from "@trigger.dev/core/v3";
import { type RuntimeEnvironmentType, WaitpointTokenStatus } from "@trigger.dev/core/v3";
import { type RunEngineVersion, type WaitpointResolver } from "@trigger.dev/database";
import { z } from "zod";
import { BasePresenter } from "./basePresenter.server";
import { CoercedDate } from "~/utils/zod";
import { AuthenticatedEnvironment } from "@internal/run-engine";
import {
WaitpointTokenListOptions,
WaitpointTokenListPresenter,
} from "./WaitpointTokenListPresenter.server";
import { ServiceValidationError } from "~/v3/services/baseService.server";
import { RunEngineVersion } from "@trigger.dev/database";
import { BasePresenter } from "./basePresenter.server";
import { type WaitpointListOptions, WaitpointListPresenter } from "./WaitpointListPresenter.server";

export const ApiWaitpointTokenListSearchParams = z.object({
export const ApiWaitpointListSearchParams = z.object({
"page[size]": z.coerce.number().int().positive().min(1).max(100).optional(),
"page[after]": z.string().optional(),
"page[before]": z.string().optional(),
Expand Down Expand Up @@ -61,9 +57,9 @@ export const ApiWaitpointTokenListSearchParams = z.object({
"filter[createdAt][to]": CoercedDate,
});

type ApiWaitpointTokenListSearchParams = z.infer<typeof ApiWaitpointTokenListSearchParams>;
type ApiWaitpointListSearchParams = z.infer<typeof ApiWaitpointListSearchParams>;

export class ApiWaitpointTokenListPresenter extends BasePresenter {
export class ApiWaitpointListPresenter extends BasePresenter {
public async call(
environment: {
id: string;
Expand All @@ -73,11 +69,13 @@ export class ApiWaitpointTokenListPresenter extends BasePresenter {
engine: RunEngineVersion;
};
},
searchParams: ApiWaitpointTokenListSearchParams
resolver: WaitpointResolver,
searchParams: ApiWaitpointListSearchParams
) {
return this.trace("call", async (span) => {
const options: WaitpointTokenListOptions = {
const options: WaitpointListOptions = {
environment,
resolver,
};

if (searchParams["page[size]"]) {
Expand Down Expand Up @@ -118,7 +116,7 @@ export class ApiWaitpointTokenListPresenter extends BasePresenter {
options.to = searchParams["filter[createdAt][to]"].getTime();
}

const presenter = new WaitpointTokenListPresenter();
const presenter = new WaitpointListPresenter();
const result = await presenter.call(options);

if (!result.success) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { type RunEngineVersion } from "@trigger.dev/database";
import { ServiceValidationError } from "~/v3/services/baseService.server";
import { BasePresenter } from "./basePresenter.server";
import { WaitpointPresenter } from "./WaitpointPresenter.server";
import { waitpointStatusToApiStatus } from "./WaitpointTokenListPresenter.server";
import { waitpointStatusToApiStatus } from "./WaitpointListPresenter.server";

export class ApiWaitpointPresenter extends BasePresenter {
public async call(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import parse from "parse-duration";
import {
Prisma,
type WaitpointResolver,
type RunEngineVersion,
type RuntimeEnvironmentType,
type WaitpointStatus,
Expand All @@ -11,10 +12,11 @@ import { BasePresenter } from "./basePresenter.server";
import { type WaitpointSearchParams } from "~/components/runs/v3/WaitpointTokenFilters";
import { determineEngineVersion } from "~/v3/engineVersion.server";
import { type WaitpointTokenStatus, type WaitpointTokenItem } from "@trigger.dev/core/v3";
import { generateWaitpointCallbackUrl } from "./WaitpointPresenter.server";

const DEFAULT_PAGE_SIZE = 25;

export type WaitpointTokenListOptions = {
export type WaitpointListOptions = {
environment: {
id: string;
type: RuntimeEnvironmentType;
Expand All @@ -23,6 +25,7 @@ export type WaitpointTokenListOptions = {
engine: RunEngineVersion;
};
};
resolver: WaitpointResolver;
// filters
id?: string;
statuses?: WaitpointTokenStatus[];
Expand All @@ -40,7 +43,7 @@ export type WaitpointTokenListOptions = {
type Result =
| {
success: true;
tokens: WaitpointTokenItem[];
tokens: (WaitpointTokenItem & { callbackUrl: string })[];
pagination: {
next: string | undefined;
previous: string | undefined;
Expand All @@ -63,9 +66,10 @@ type Result =
filters: undefined;
};

export class WaitpointTokenListPresenter extends BasePresenter {
export class WaitpointListPresenter extends BasePresenter {
public async call({
environment,
resolver,
id,
statuses,
idempotencyKey,
Expand All @@ -76,7 +80,7 @@ export class WaitpointTokenListPresenter extends BasePresenter {
direction = "forward",
cursor,
pageSize = DEFAULT_PAGE_SIZE,
}: WaitpointTokenListOptions): Promise<Result> {
}: WaitpointListOptions): Promise<Result> {
const engineVersion = await determineEngineVersion({ environment });
if (engineVersion === "V1") {
return {
Expand Down Expand Up @@ -165,7 +169,7 @@ export class WaitpointTokenListPresenter extends BasePresenter {
${sqlDatabaseSchema}."Waitpoint" w
WHERE
w."environmentId" = ${environment.id}
AND w.type = 'MANUAL'
AND w.resolver = ${resolver}::"WaitpointResolver"
-- cursor
${
cursor
Expand Down Expand Up @@ -250,7 +254,7 @@ export class WaitpointTokenListPresenter extends BasePresenter {
const firstToken = await this._replica.waitpoint.findFirst({
where: {
environmentId: environment.id,
type: "MANUAL",
resolver,
},
});

Expand All @@ -263,6 +267,7 @@ export class WaitpointTokenListPresenter extends BasePresenter {
success: true,
tokens: tokensToReturn.map((token) => ({
id: token.friendlyId,
callbackUrl: generateWaitpointCallbackUrl(token.id),
status: waitpointStatusToApiStatus(token.status, token.outputIsError),
completedAt: token.completedAt ?? undefined,
timeoutAt: token.completedAfter ?? undefined,
Expand Down
16 changes: 15 additions & 1 deletion apps/webapp/app/presenters/v3/WaitpointPresenter.server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ import { isWaitpointOutputTimeout, prettyPrintPacket } from "@trigger.dev/core/v
import { logger } from "~/services/logger.server";
import { BasePresenter } from "./basePresenter.server";
import { type RunListItem, RunListPresenter } from "./RunListPresenter.server";
import { waitpointStatusToApiStatus } from "./WaitpointTokenListPresenter.server";
import { waitpointStatusToApiStatus } from "./WaitpointListPresenter.server";
import { WaitpointId } from "@trigger.dev/core/v3/isomorphic";
import { env } from "~/env.server";

export type WaitpointDetail = NonNullable<Awaited<ReturnType<WaitpointPresenter["call"]>>>;

Expand Down Expand Up @@ -35,6 +37,7 @@ export class WaitpointPresenter extends BasePresenter {
completedAfter: true,
completedAt: true,
createdAt: true,
resolver: true,
connectedRuns: {
select: {
friendlyId: true,
Expand Down Expand Up @@ -83,6 +86,11 @@ export class WaitpointPresenter extends BasePresenter {
return {
id: waitpoint.friendlyId,
type: waitpoint.type,
resolver: waitpoint.resolver,
callbackUrl:
waitpoint.resolver === "HTTP_CALLBACK"
? generateWaitpointCallbackUrl(waitpoint.friendlyId)
: undefined,
status: waitpointStatusToApiStatus(waitpoint.status, waitpoint.outputIsError),
idempotencyKey: waitpoint.idempotencyKey,
userProvidedIdempotencyKey: waitpoint.userProvidedIdempotencyKey,
Expand All @@ -100,3 +108,9 @@ export class WaitpointPresenter extends BasePresenter {
};
}
}

export function generateWaitpointCallbackUrl(waitpointId: string) {
return `${
env.API_ORIGIN ?? env.APP_ORIGIN
}/api/v1/waitpoints/http-callback/${WaitpointId.toFriendlyId(waitpointId)}/callback`;
}
Loading