Http Utils

When calling ServiceStack services the recommended way is to use any of the C# Service Clients that have an optimal DRY and typed API. However often when doing server programming you need to consume 3rd Party HTTP APIs, unfortunately the built-in way to do this in .NET doesn't make for a good development experience.

One of the legacy classes inherited from the early .NET days, that's still in active use today is WebRequest. It's an example of a class that's both versatile but also suffers from exposing an out-dated and unpleasant API for your application code to bind to.

Rather than taking the normal .NET approach of wrapping it inside a suite of proxy and abstraction classes, we prefer to instead wrap any unnecessarily verbose APIs behind extension methods encapsulating common access patterns behind a DRY, terse and readable chained API without any loss of flexibility since the underlying WebRequest remains accessible whenever it's needed.

The PocoPower project shows some good examples of what this looks like in Practice. Here's how you can retrieve a typed list of GitHub User Repos from GitHub's JSON REST API:

List<GithubRepo> userRepos = "https://api.github.com/users/{0}/repos".Fmt(user)
    .GetJsonFromUrl()
    .FromJson<List<GithubRepo>>();

Url Extensions

You can make use of the accompanying String Extensions to programatically construct a url as seen in this Twitter API example:

var url = "http://api.twitter.com/1/statuses/user_timeline.json?screen_name={0}".Fmt(name);
if (sinceId != null)
    url = url.AddQueryParam("since_id", sinceId);
if (maxId != null)
    url = url.AddQueryParam("max_id", maxId);

var tweets = url.GetJsonFromUrl()
    .FromJson<List<Tweet>>();

In both these cases it uses WebRequest to make a HTTP GET request asking for the "application/json" Content-Type, that's preferably compressed with gzip or deflate encoding (if the remote web server supports it).

Alternative Content-Type

In addition to GetJsonFromUrl there's also GetXmlFromUrl covering the 2 widely used content-types used for data containers:

List<User> users = "http://example.org/xml-rpc/users"
    .GetStringFromUrl(acceptContentType:"application/xml")
    .FromXml<List<User>>();

For any other Content-Type you can specify it with the optional acceptContentType param:

var csv = "http://example.org/users.csv".GetStringFromUrl(acceptContentType:"text/csv");

Customizing the WebRequest

Although most extension methods start on string urls, you can customize the HttpWebRequest used to make the request by specifying a requestFilter. e.g:

var json = "http://example.org/users".GetJsonFromUrl(requestFilter:httpReq => {
    webReq.Headers["X-Api-Key"] = apiKey;
});

Parsing Custom Responses

This also works for Response Filters as well where if you need access to the Response HTTP Headers as well as the body you can add a callback for the response filter:

List<GithubRepo> userRepos = "https://api.github.com/users/{0}/repos".Fmt(user)
    .GetJsonFromUrl(responseFilter: httpRes => {
        var remaining = httpRes.Headers["X-Api-Remaining"];
    })
    .FromJson<List<GithubRepo>>();

Downloading Raw Content

Use the GetStringFromUrl extension to download raw text:

string csv = "http://example.org/sales.csv".GetStringFromUrl();

and the GetBytesFromUrl extension to download raw bytes:

byte[] imgBytes = "http://example.org/photo.jpg".GetBytesFromUrl();

POSTing data

Another common HTTP Request is to POST data to REST APIs. The most common way to post data to HTTP APIs is to post x-www-form-urlencoded key value pairs which you can do with:

var response = "http://example.org/login"
    .PostToUrl("Username=mythz&Password=password");

Or using a POCO Type:

var response = "http://example.org/login"
    .PostToUrl(new Login { Username="mythz", Password="password" });

An Anonymous Type:

var response = "http://example.org/login"
    .PostToUrl(new { Username="mythz", Password="password" });

Or even a Dictionary:

var login = new Dictionary<string,string> { 
    {"Username","mythz"}, {"Password","password"} };
var response = "http://example.org/login".PostToUrl(login);

Although POST'ing other Content-Types are also easily supported. An example using JSON:

Either as any serializable JSON object:

var response = "http://example.org/login"
    .PostJsonToUrl(new Login { Username="mythz", Password="password" });

Or as a raw JSON string:

var response = "http://example.org/login"
    .PostJsonToUrl("{\"Username\":\"mythz",\"Password\":\"p@ssword\"}");

And an example of sending any other arbitrary content types:

var response = "http://example.org/login"
  .PostToUrl("<User>mythz</User><Pass>p@ssword</Pass>", contentType:"application/xml");

The above API's also apply to PUT data as well by using the PutToUrl extension methods.

Uploading Files

In the ServiceStack.ServiceClient.Web namespace there are more HTTP extensions available including the UploadFile extension methods to upload files using multi-part/formdata:

var httpRes = "http://example.org/upload"
    .UploadFile(new FileInfo("/path/to/file.xml"), "application/xml");

Exception handling

Exception handling is another area we can DRY using extension methods. Rather than including the boilerplate required to find the underlying issue for every request, we provide typed APIs on Exception for common HTTP Faults:

try 
{
    var response = "http://example.org/fault".GetStringFromUrl();
} 
catch (Exception ex) 
{
    var knownError = ex.IsBadRequest() || ex.IsNotFound() || ex.IsUnauthorized() 
        || ex.IsForbidden() || ex.IsInternalServerError();

    var isAnyRedirectionError = ex.IsAny300();
    var isAnyClientError = ex.IsAny400();
    var isAnyServerError = ex.IsAny500();

    HttpStatusCode? errorStatus = ex.GetStatus();
}

HTTP API Reference

The above description should give you a good idea of how to make use of the APIs, although for a more complete reference we'll post the full signatures here.

Most of the APIs are located in the ServiceStack.Text namespace:

string GetJsonFromUrl(this string url, Action<HttpWebRequest> requestFilter=null,
  Action<HttpWebResponse> responseFilter=null)

string GetXmlFromUrl(this string url, Action<HttpWebRequest> requestFilter=null, 
  Action<HttpWebResponse> responseFilter=null)

string GetStringFromUrl(this string url, string acceptContentType = "*/*", 
  Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string PostToUrl(this string url, string formData=null, string acceptContentType="*/*",
  Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string PostToUrl(this string url, object formData = null, string acceptContentType="*/*",
  Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string PostJsonToUrl(this string url, string json,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PostJsonToUrl(this string url, object data,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PostXmlToUrl(this string url, string xml,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PostXmlToUrl(this string url, object data,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PutToUrl(this string url, string formData=null, string acceptContentType="*/*",
  Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string PutToUrl(this string url, object formData = null, string acceptContentType = "*/*",
  Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string PutJsonToUrl(this string url, string json,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PutJsonToUrl(this string url, object data,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PutXmlToUrl(this string url, string xml,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string PutXmlToUrl(this string url, object data,
    Action<HttpWebRequest> requestFilter = null, Action<HttpWebResponse> responseFilter = null)

string DeleteFromUrl(this string url, string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string OptionsFromUrl(this string url, string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string HeadFromUrl(this string url, string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

string SendStringToUrl(this string url, string method = null,
    string requestBody = null, string contentType = null, string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

byte[] GetBytesFromUrl(this string url, string acceptContentType = "*/*", 
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

byte[] PostBytesToUrl(this string url, byte[] requestBody = null, string contentType = null, 
    string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

byte[] PutBytesToUrl(this string url, byte[] requestBody = null, string contentType = null, 
    string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

byte[] SendBytesToUrl(this string url, string method = null,
    byte[] requestBody = null, string contentType = null, string acceptContentType = "*/*",
    Action<HttpWebRequest> requestFilter=null, Action<HttpWebResponse> responseFilter=null)

bool IsAny200(this Exception ex)
bool IsAny300(this Exception ex)
bool IsAny400(this Exception ex)
bool IsAny500(this Exception ex)
bool IsBadRequest(this Exception ex)
bool IsNotFound(this Exception ex)
bool IsUnauthorized(this Exception ex)
bool IsForbidden(this Exception ex)
bool IsInternalServerError(this Exception ex)

HttpStatusCode? GetResponseStatus(this string url)
HttpStatusCode? GetStatus(this Exception ex)
HttpStatusCode? GetStatus(this WebException webEx)

bool HasStatus(this WebException webEx, HttpStatusCode statusCode)

Whilst some additional HTTP APIs can be found in the ServiceStack.ServiceClient.Web namespace:

HttpWebResponse GetErrorResponse(this string url)

WebResponse PostFileToUrl(this string url, FileInfo uploadFileInfo, string uploadFileMimeType,
    string acceptContentType = null, Action<HttpWebRequest> requestFilter = null)

WebResponse UploadFile(this WebRequest webRequest, FileInfo uploadFileInfo, string uploadFileMimeType)

UploadFile(this WebRequest webRequest, Stream fileStream, string fileName, string mimeType,
    string acceptContentType = null, Action<HttpWebRequest> requestFilter = null)

void UploadFile(this WebRequest webRequest, Stream fileStream, string fileName)

Which is located in the ServiceStack.Common NuGet package