Support custom protocol callback URLs

[ranmocy]

Goals

After signin success, allow the app to redirect to a custom protocol URL like x-my-app://localhost, so that native app like Electron could handle it directly.

Non-Goals

Background

In assertConfig it would throw exception if the callback URL is not HTTP(S) protocol.

next-auth/packages/core/src/lib/assert.ts

Lines 84 to 102 in 5d93a8b

	if (callbackUrlParam && !isValidHttpUrl(callbackUrlParam, url.origin)) {
	return new InvalidCallbackUrl(
	`Invalid callback URL. Received: ${callbackUrlParam}`
	)
	}
	const { callbackUrl: defaultCallbackUrl } = defaultCookies(
	options.useSecureCookies ?? url.protocol === "https:"
	)
	const callbackUrlCookie =
	request.cookies?.[
	options.cookies?.callbackUrl?.name ?? defaultCallbackUrl.name
	]
	if (callbackUrlCookie && !isValidHttpUrl(callbackUrlCookie, url.origin)) {
	return new InvalidCallbackUrl(
	`Invalid callback URL. Received: ${callbackUrlCookie}`
	)
	}

Proposal

Remove check of isValidHttpUrl to allow arbitrary callback URLs. Optionally allow the app itself to provide the validator.