Merge pull request #52 from thedodd/33-proxy

Implement proxy system

Author: thedodd

CHANGELOG.md

@@ -4,6 +4,13 @@ This changelog follows the patterns described here: https://keepachangelog.com/e
 
 ## Unreleased
 
+## 0.5.0
+### added
+- Added support for proxying requests to arbitrary HTTP backends.
+    - This includes a basic CLI based config.
+    - This also includes a more robust `Trunk.toml` config which allows for specifying multiple proxies.
+- Added a `trunk config show` subcommand. This command will pretty-print Trunk's final runtime config based on any config file & env vars. This can be useful for debugging & testing.
+
 ## 0.4.0
 ### added
 - In addition to CLI arguments and options, Trunk now supports layered configuration via `Trunk.toml` & environment variables.

CONTRIBUTING.md

@@ -2,6 +2,12 @@ contributing
 ============
 We are all living beings, and what is most important is that we respect each other and work together. If you can not uphold this simple standard, then your contributions are not welcome.
 
+## hacking
+Just a few simple items to keep in mind as you hack.
+
+- Pull request early and often. This helps to let others know what you are working on. **Please use Github's Draft PR mechansim** if your PR is not yet ready for review.
+- Remember to update the `CHANGELOG.md` once you believe your work is nearing completion.
+
 ## linting
 We are using clippy & rustfmt. Clippy is SO GREAT! Rustfmt ... has a lot more growing to do; however, we are using it for uniformity.
 

Cargo.lock

@@ -1,6 +1,6 @@
 [package]
 name = "trunk"
-version = "0.4.0"
+version = "0.5.0"
 edition = "2018"
 description = "Build, bundle & ship your Rust WASM application to the web."
 license = "MIT/Apache-2.0"
@@ -19,6 +19,7 @@ console = "0.12.0"
 envy = "0.4.1"
 fs_extra = "1.2.0"
 futures = "0.3.5"
+http-types = "2.4.0"
 indicatif = "0.15.0"
 nipper = "0.1.8"
 notify = "4.0.15"
@@ -27,7 +28,8 @@ sass-rs = "0.2.2"
 seahash = "4.0.1"
 serde = { version="1", features=["derive"] }
 structopt = "0.3.16"
-tide = "0.13.0"
+surf = "2.0.0-alpha.5"
+tide = { version="0.13.0", features=["unstable"] }
 toml = "0.5.6"
 
 [profile.release]

Cargo.toml

@@ -111,6 +111,29 @@ Trunk environment variables mirror the `Trunk.toml` config schema. All Trunk env
 ### cli arguments & options
 The final configuration layer is the CLI itself. Any arguments / options provided on the CLI will take final precedence over any other config layer.
 
+## proxy
+Trunk ships with a built-in proxy which can be enabled when running `trunk serve`. There are two ways to configure the proxy, each discussed below. All Trunk proxies will transparently pass along the request body, headers, and query parameters to the proxy backend.
+
+### proxy cli flags
+The `trunk serve` command accepts two proxy related flags.
+
+`--proxy-backend` specifies the URL of the backend server to which requests should be proxied. The URI segment of the given URL will be used as the path on the Trunk server to handle proxy requests. E.G., `trunk serve --proxy-backend=http://localhost:9000/api/` will proxy any requests received on the path `/api/` to the server listening at `http://localhost:9000/api/`. Further path segments or query parameters will be seamlessly passed along.
+
+`--proxy-rewrite` specifies an alternative URI on which the Trunk server is to listen for proxy requests. Any requests received on the given URI will be rewritten to match the URI of the proxy backend, effectively stripping the rewrite prefix. E.G., `trunk serve --proxy-backend=http://localhost:9000/ --proxy-rewrite=/api/` will proxy any requests received on `/api/` over to `http://localhost:9000/` with the `/api/` prefix stripped from the request, while everything following the `/api/` prefix will be left unchanged.
+
+### config file
+The `Trunk.toml` config file accepts multiple `[[proxy]]` sections, which allows for multiple proxies to be configured. Each section requires at least the `backend` field, and optionally accepts the `rewrite` field, both corresponding to the `--proxy-*` CLI flags discussed above.
+
+As it is with other Trunk config, a proxy declared via CLI will take final precedence and will cause any config file proxies to be ignored, even if there are multiple proxies declared in the config file.
+
+The following is a snippet from the `Trunk.toml` file in this repo:
+
+```toml
+[[proxy]]
+rewrite = "/api/v1/"
+backend = "http://localhost:9000/"
+```
+
 ## contributing
 Anyone and everyone is welcome to contribute! Please review the [CONTRIBUTING.md](./CONTRIBUTING.md) document for more details. The best way to get started is to find an open issue, and then start hacking on implementing it. Letting other folks know that you are working on it, and sharing progress is a great approach. Open pull requests early and often, and please use Github's draft pull request feature.
 

README.md

@@ -27,3 +27,19 @@ open = false
 dist = "dist"
 # Optionally perform a cargo clean.
 cargo = false
+
+## proxy
+# Proxies are optional, and default to `None`.
+# Proxies are only run as part of the `trunk serve` command.
+
+[[proxy]]
+# This proxy example has a backend and a rewrite field. Requests received on `rewrite` will be
+# proxied to the backend after rewriting the `rewrite` prefix to the `backend`'s URI prefix.
+# E.G., `/api/v1/resource/x/y/z` -> `/resource/x/y/z`
+rewrite = "/api/v1/"
+backend = "http://localhost:9000/"
+
+[[proxy]]
+# This proxy specifies only the backend, which is the only required field. In this example,
+# request URIs are not modified when proxied.
+backend = "http://localhost:9000/api/v2/"

Trunk.toml

@@ -0,0 +1,35 @@
+use std::path::PathBuf;
+
+use anyhow::Result;
+use structopt::StructOpt;
+
+use crate::config::ConfigOpts;
+
+/// Trunk config controls.
+#[derive(Clone, Debug, StructOpt)]
+#[structopt(name = "config")]
+pub struct Config {
+    #[structopt(subcommand)]
+    action: ConfigSubcommands,
+}
+
+impl Config {
+    pub async fn run(self, config: Option<PathBuf>) -> Result<()> {
+        // NOTE WELL: if we ever add additional subcommands, refactor this to match the pattern
+        // used in main, which is much more scalable. This is faster to code, and will not force
+        // incompatibility when new commands are added.
+        match self.action {
+            ConfigSubcommands::Show => {
+                let cfg = ConfigOpts::full(config).await?;
+                println!("{:#?}", cfg);
+            }
+        }
+        Ok(())
+    }
+}
+
+#[derive(Clone, Debug, StructOpt)]
+enum ConfigSubcommands {
+    /// Show Trunk's current config pre-CLI.
+    Show,
+}

src/cmd/config.rs

@@ -1,4 +1,5 @@
 pub mod build;
 pub mod clean;
+pub mod config;
 pub mod serve;
 pub mod watch;

src/cmd/mod.rs

@@ -8,6 +8,7 @@ use std::path::PathBuf;
 use std::sync::Arc;
 
 use anyhow::Result;
+use http_types::Url;
 use serde::Deserialize;
 use structopt::StructOpt;
 
@@ -69,15 +70,40 @@ pub struct RtcServe {
     pub port: u16,
     /// Open a browser tab once the initial build is complete.
     pub open: bool,
+    /// A URL to which requests will be proxied.
+    pub proxy_backend: Option<Url>,
+    /// The URI on which to accept requests which are to be rewritten and proxied to backend.
+    pub proxy_rewrite: Option<String>,
+    /// Any proxies configured to run along with the server.
+    pub proxies: Option<Vec<ConfigOptsProxy>>,
 }
 
-impl From<(CargoMetadata, ConfigOptsBuild, ConfigOptsWatch, ConfigOptsServe)> for RtcServe {
-    fn from((manifest, build_opts, watch_opts, opts): (CargoMetadata, ConfigOptsBuild, ConfigOptsWatch, ConfigOptsServe)) -> Self {
+impl
+    From<(
+        CargoMetadata,
+        ConfigOptsBuild,
+        ConfigOptsWatch,
+        ConfigOptsServe,
+        Option<Vec<ConfigOptsProxy>>,
+    )> for RtcServe
+{
+    fn from(
+        (manifest, build_opts, watch_opts, opts, proxies): (
+            CargoMetadata,
+            ConfigOptsBuild,
+            ConfigOptsWatch,
+            ConfigOptsServe,
+            Option<Vec<ConfigOptsProxy>>,
+        ),
+    ) -> Self {
         let watch = Arc::new(RtcWatch::from((manifest, build_opts, watch_opts)));
         Self {
             watch,
             port: opts.port.unwrap_or(8080),
             open: opts.open,
+            proxy_backend: opts.proxy_backend,
+            proxy_rewrite: opts.proxy_rewrite,
+            proxies,
         }
     }
 }
@@ -142,6 +168,14 @@ pub struct ConfigOptsServe {
     #[structopt(long)]
     #[serde(default)]
     pub open: bool,
+    /// A URL to which requests will be proxied [default: None]
+    #[structopt(long = "proxy-backend")]
+    #[serde(default)]
+    pub proxy_backend: Option<Url>,
+    /// The URI on which to accept requests which are to be rewritten and proxied to backend [default: None]
+    #[structopt(long = "proxy-rewrite")]
+    #[serde(default)]
+    pub proxy_rewrite: Option<String>,
 }
 
 /// Config options for the serve system.
@@ -156,13 +190,30 @@ pub struct ConfigOptsClean {
     pub cargo: bool,
 }
 
+/// Config options for building proxies.
+///
+/// NOTE WELL: this configuration type is different from the others inasmuch as it is only used
+/// when parsing the `Trunk.toml` config file. It is not intended to be configured via CLI or env vars.
+#[derive(Clone, Debug, Deserialize)]
+pub struct ConfigOptsProxy {
+    /// The URL of the backend to which requests are to be proxied.
+    pub backend: Url,
+    /// An optional URI prefix which is to be used as the base URI for proxying requests, which
+    /// defaults to the URI of the backend.
+    ///
+    /// When a value is specified, requests received on this URI will have this URI segment replaced
+    /// with the URI of the `backend`.
+    pub rewrite: Option<String>,
+}
+
 /// A model of all potential configuration options for the Trunk CLI system.
 #[derive(Clone, Debug, Default, Deserialize)]
 pub struct ConfigOpts {
     pub build: Option<ConfigOptsBuild>,
     pub watch: Option<ConfigOptsWatch>,
     pub serve: Option<ConfigOptsServe>,
     pub clean: Option<ConfigOptsClean>,
+    pub proxy: Option<Vec<ConfigOptsProxy>>,
 }
 
 impl ConfigOpts {
@@ -198,7 +249,13 @@ impl ConfigOpts {
         let watch_opts = serve_layer.watch.unwrap_or_default();
         let serve_opts = serve_layer.serve.unwrap_or_default();
         let manifest = CargoMetadata::new(&build_opts.manifest).await?;
-        Ok(Arc::new(RtcServe::from((manifest, build_opts, watch_opts, serve_opts))))
+        Ok(Arc::new(RtcServe::from((
+            manifest,
+            build_opts,
+            watch_opts,
+            serve_opts,
+            serve_layer.proxy,
+        ))))
     }
 
     /// Extract the runtime config for the clean system based on all config layers.
@@ -209,6 +266,11 @@ impl ConfigOpts {
         Ok(Arc::new(RtcClean::from(clean_opts)))
     }
 
+    /// Return the full configuration based on config file & environment variables.
+    pub async fn full(config: Option<PathBuf>) -> Result<Self> {
+        Self::file_and_env_layers(config)
+    }
+
     fn cli_opts_layer_build(cli: ConfigOptsBuild, cfg_base: Self) -> Self {
         let opts = ConfigOptsBuild {
             target: cli.target,
@@ -222,6 +284,7 @@ impl ConfigOpts {
             watch: None,
             serve: None,
             clean: None,
+            proxy: None,
         };
         Self::merge(cfg_base, cfg_build)
     }
@@ -233,6 +296,7 @@ impl ConfigOpts {
             watch: Some(opts),
             serve: None,
             clean: None,
+            proxy: None,
         };
         Self::merge(cfg_base, cfg)
     }
@@ -241,12 +305,15 @@ impl ConfigOpts {
         let opts = ConfigOptsServe {
             port: cli.port,
             open: cli.open,
+            proxy_backend: cli.proxy_backend,
+            proxy_rewrite: cli.proxy_rewrite,
         };
         let cfg = ConfigOpts {
             build: None,
             watch: None,
             serve: Some(opts),
             clean: None,
+            proxy: None,
         };
         Self::merge(cfg_base, cfg)
     }
@@ -261,6 +328,7 @@ impl ConfigOpts {
             watch: None,
             serve: None,
             clean: Some(opts),
+            proxy: None,
         };
         Self::merge(cfg_base, cfg)
     }
@@ -293,6 +361,7 @@ impl ConfigOpts {
             watch: Some(watch),
             serve: Some(serve),
             clean: Some(clean),
+            proxy: None,
         })
     }
 
@@ -325,6 +394,8 @@ impl ConfigOpts {
             (None, None) => None,
             (Some(val), None) | (None, Some(val)) => Some(val),
             (Some(l), Some(mut g)) => {
+                g.proxy_backend = g.proxy_backend.or(l.proxy_backend);
+                g.proxy_rewrite = g.proxy_rewrite.or(l.proxy_rewrite);
                 g.port = g.port.or(l.port);
                 // NOTE: this can not be disabled in the cascade.
                 if l.open {
@@ -345,6 +416,11 @@ impl ConfigOpts {
                 Some(g)
             }
         };
+        greater.proxy = match (lesser.proxy.take(), greater.proxy.take()) {
+            (None, None) => None,
+            (Some(val), None) | (None, Some(val)) => Some(val),
+            (Some(_), Some(g)) => Some(g), // No meshing/merging. Only take the greater value.
+        };
         greater
     }
 }

src/config.rs

@@ -2,6 +2,7 @@ mod build;
 mod cmd;
 mod common;
 mod config;
+mod proxy;
 mod serve;
 mod watch;
 
@@ -38,6 +39,7 @@ impl Trunk {
             TrunkSubcommands::Clean(inner) => inner.run(self.config).await,
             TrunkSubcommands::Serve(inner) => inner.run(self.config).await,
             TrunkSubcommands::Watch(inner) => inner.run(self.config).await,
+            TrunkSubcommands::Config(inner) => inner.run(self.config).await,
         }
     }
 }
@@ -48,4 +50,5 @@ enum TrunkSubcommands {
     Clean(cmd::clean::Clean),
     Serve(cmd::serve::Serve),
     Watch(cmd::watch::Watch),
+    Config(cmd::config::Config),
 }