Need an API for flushing additional stuff at the end of PipeWriter.

[AwsomeCode]

In Aes encryption we need to add padding at the end of stream by calling CryptoStream.FlushFinalBlock() https://stackoverflow.com/a/40564155/2949571
I can't find any api to do that via PipeWriter/BodyWriter in asp.net core.

Issue is that client side I am receiving encrypted data without padding. So client side is not able to decrypt the data.
@davidfowl

[davidfowl]

Can you clarify what exactly you're trying to do with a running code sample?

[AwsomeCode]

I am following https://tudip.com/blog-post/how-to-securely-transfer-web-api-data-in-asp-net-core/ blog for Encrypting and Decrypting all the incoming and outgoing traffic using Middleware.

While encrypting data in Aes data need's to be a fixed size block.
For that we need to add padding to the final block to match the size of other blocks.
So Padding can only be set by calling CryptoStream.FlushFinalBlock() at the end of all the Flush by PipeWriter/BodyWriter.

using System;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Hosting;
using System.Security.Cryptography;
using System.IO;

namespace simpleWebApp
{
    public class Startup
    {
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.Use( async (httpContext, next) =>
            {
                httpContext.Response.Body = EncryptStream(httpContext.Response.Body);
                httpContext.Request.Body = DecryptStream(httpContext.Request.Body);
                await next();
            });
            app.Run(async context => 
            {
                var str = "{\"pageViews\":37,\"subscriberCount\":2}";
                Console.WriteLine(str);
                await context.Response.WriteAsync(str);
            });
        }

        private static CryptoStream EncryptStream(Stream responseStream)
        {
            Aes aes = GetEncryptionAlgorithm();
            CryptoStream base64EncodedStream = new CryptoStream(responseStream, new ToBase64Transform(), CryptoStreamMode.Write);
            CryptoStream cryptoStream = new CryptoStream(base64EncodedStream, aes.CreateEncryptor(aes.Key, aes.IV), CryptoStreamMode.Write);
           //CryptoStream.FlushFinalBlock() // this will not work because this needs to be push at the end of stream.
            return cryptoStream;
        }

        private static Stream DecryptStream(Stream cipherStream)
        {
            Aes aes = GetEncryptionAlgorithm();
            CryptoStream base64DecodedStream = new CryptoStream(cipherStream, new FromBase64Transform(FromBase64TransformMode.IgnoreWhiteSpaces), CryptoStreamMode.Read);
            CryptoStream decryptedStream = new CryptoStream(base64DecodedStream, aes.CreateDecryptor(aes.Key, aes.IV), CryptoStreamMode.Read);
            return decryptedStream;
        }

        private static Aes GetEncryptionAlgorithm()
        {
            AesManaged aes = new AesManaged
            {   
                Padding = PaddingMode.PKCS7,
                Key = Convert.FromBase64String("73kczzrPtnn5GXxQtOI6m4AewK34F4IkT/yaeYZxr+M="),
                IV = Convert.FromBase64String("b8LAfmzY8WxGNrZeTye4hw=="),
            };
            return aes;
        }
    }
}

[davidfowl]

Why aren't you using https instead?

[AwsomeCode]

To protect our JSON schema.
We are using HTTPS but it can't protect our JSON schema from our authenticated user.
Our end client is Mobile App. So we are going to bundle the decryption key within the app.

[davidfowl]

I don't know if this approach makes any sense (cc @blowdart), but to answer your question about when to call FlushFinalBlock:

• You can call DisposeAsync on the CryptoStream after the call to next()
• You can call FlushFinalBlock async on the CryptoStream after to next()

app.Use(async (httpContext, next) =>
{
    var oldResponse = httpContext.Response.Body;
    var oldRequest = httpContext.Request.Body;

    try
    {
       await using var newResponseBody = EncryptStream(oldResponse);
       await using var newRequestBody = DecryptStream(oldRequest );
        
        httpContext.Response.Body = newResponseBody;
        httpContext.Request.Body = newRequestBody;

        await next();
    }
    finally
    {
        httpContext.Response.Body = oldResponse;
        httpContext.Request.Body = oldRequest;
    }
});

The code above made some tweaks to the original code to dispose the crypto stream when complete (using IAsyncDisposable and await using)

[AwsomeCode]

This is working. Only typo error is
await using var newRequestBody = DecryptStream(oldResponse);
to
await using var newRequestBody = DecryptStream(oldRequest);

[AwsomeCode]

@davidfowl it worked with simple scenario. But not working with endpoints.

[AwsomeCode]

Getting exception after calling

await next();
await newResponseBody.DisposeAsync();
And Same for
newResponseBody.FlushFinalBlock();

fail: Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HM2RLFCNJK7R", Request id "0HM2RLFCNJK7R:00000001": An unhandled exception was thrown by the application.
System.ArgumentException: Stream was not readable.
   at System.Security.Cryptography.CryptoStream..ctor(Stream stream, ICryptoTransform transform, CryptoStreamMode mode, Boolean leaveOpen)
   at System.Security.Cryptography.CryptoStream..ctor(Stream stream, ICryptoTransform transform, CryptoStreamMode mode)
   at simpleWebApp.Startup.DecryptStream(Stream cipherStream) in C:\p\youtube\simpleWebApp\Startup.cs:line 69
   at simpleWebApp.Startup.<>c.<<Configure>b__1_0>d.MoveNext() in C:\p\youtube\simpleWebApp\Startup.cs:line 36
--- End of stack trace from previous location where exception was thrown ---
   at simpleWebApp.Startup.<>c.<<Configure>b__1_0>d.MoveNext() in C:\p\youtube\simpleWebApp\Startup.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
Microsoft.AspNetCore.Server.Kestrel: Error: Connection id "0HM2RLFCNJK7R", Request id "0HM2RLFCNJK7R:00000001": An unhandled exception was thrown by the application.

System.ArgumentException: Stream was not readable.
   at System.Security.Cryptography.CryptoStream..ctor(Stream stream, ICryptoTransform transform, CryptoStreamMode mode, Boolean leaveOpen)
   at System.Security.Cryptography.CryptoStream..ctor(Stream stream, ICryptoTransform transform, CryptoStreamMode mode)
   at simpleWebApp.Startup.DecryptStream(Stream cipherStream) in C:\p\youtube\simpleWebApp\Startup.cs:line 69
   at simpleWebApp.Startup.<>c.<<Configure>b__1_0>d.MoveNext() in C:\p\youtube\simpleWebApp\Startup.cs:line 36
--- End of stack trace from previous location where exception was thrown ---
   at simpleWebApp.Startup.<>c.<<Configure>b__1_0>d.MoveNext() in C:\p\youtube\simpleWebApp\Startup.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

[davidfowl]

I'm not sure what the issue is there, the request stream is readable so maybe use a debugger to figure out which stream is causing the problems.