Unit Testing the IIS URL Rewrite Module

Books I read

See all recommendations

Overview

I can't count on two hands the amount of times I've messed up some IIS redirect rule on our sites. "We need all these 50 old top traffic drivers from the old site, and of course we want five domains per country in case someone mistypes it and Google don't show us as first hit.
Oh, and by the way, everybody should be able to use these three slugs for our new campaigs."

Bloody marketeers!

Wouldn't it be nice if we could write some automated tests that verifies that we've configured the UrlRewriteModule correctly? I've been half-heartedly looking for a way to do this with the proprietary .NET Framework IIS Module, but recently realized while researching it that it's a public middleware in .NET Core.
This means we can start to write unit tests against it, and verify that stuff we want to use with our old .NET Framework sites will work even if we test for - and redirect to domains.

Test dependencies

Since I'm testing the config of a .NET Framework app, I just add a separate .NET Core 6 NUnit test project. Its sole purpose is verifying these redirects.

All the dependencies are nuget packages, but I just went and used some fixed paths to the SDK for now. Here are the test project dependencies:

<ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
    <PackageReference Include="Moq" Version="4.18.2" />
    <PackageReference Include="NUnit" Version="3.13.3" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
    <PackageReference Include="NUnit.Analyzers" Version="3.3.0" />
    <PackageReference Include="Verify.NUnit" Version="17.10.2" />
</ItemGroup>

<ItemGroup>
    <Reference Include="Microsoft.AspNetCore.Http">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.AspNetCore.Http.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.AspNetCore.Http.Abstractions">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.AspNetCore.Http.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.AspNetCore.Rewrite">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.AspNetCore.Rewrite.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.AspNetCore.Http.Features">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.AspNetCore.Http.Features.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.AspNetCore.Hosting.Abstractions">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.AspNetCore.Hosting.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Features">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.Extensions.Features.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Hosting.Abstractions">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.Extensions.Hosting.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Logging.Abstractions">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.Extensions.Logging.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.FileProviders.Abstractions">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.Extensions.FileProviders.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Options">
        <HintPath>C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\6.0.9\Microsoft.Extensions.Options.dll</HintPath>
    </Reference>
</ItemGroup>

Now that we have them, we can start setting up a totally mocked HttpContext and look at what the IIS UrlRewritingModule middleware does.

The middleware

In order to use the IIS UrlRewriteModule in ASP.NET Core we do app.UseRewriter(RewriteOptions) and on the RewriteOptions we do .AddIISUrlRewrite(FileProvider, "web.config"). All these two things do is camouflaging a middleware and parsing of the config.
What we need in our test is this:

// omitted quite a few web hosting mocks
var reader = // some way to read our config as a web config file
var options = new RewriteOptions().AddIISUrlRewrite(reader);
var middleware = new RewriteMiddleware(_ => Task.CompletedTask, Mock.Of<IWebHostEnvironment>(), loggerFactory, new OptionsWrapper<RewriteOptions>(options))
middleware.Invoke(httpContext);
Verify(state);

All necessary code is at the bottom of this post.

Reading the configuration

We usually put our rewrite rules in a separate file in the web project. It's linked from the web.config file using the configSource attribute as such:

<system.webServer>
    <rewrite>
        <rules configSource="config\RewriteRules.config" />
    </rewrite>
</system.webServer>

To be frank it's actually done on publish using a transform file, so we don't have it configured when working locally.
The rules are then managed in a file looking like this:

<?xml version="1.0" encoding="utf-8" ?>
<rules>
    <rule enabled="true" name="test">
        <match url="(.*)" />
        <conditions logicalGrouping="MatchAll">
            <add input="{HTTP_URL}" pattern=".*abc.*" matchType="Pattern"  />
        </conditions>
        <action type="Redirect" redirectType="Found" url="/tada" />
    </rule>
</rules>

However the AddIISUrlRewrite extension for the RewriteOptions require a stream of a full web.config file. I've just hacked together a small factory method that wraps our config in a slim web.config envelope:

var path = Path.Join(AppDomain.CurrentDomain.BaseDirectory, @"..\..\..\..\Customer.Web\config\rewriterules.config");
var contents = File.ReadAllText(path)
    .Replace("<?xml version=\"1.0\" encoding=\"utf-8\" ?>", "");
var fullContents = "<?xml version=\"1.0\" encoding=\"utf-8\" ?><configuration><system.webServer><rewrite>" + 
                    contents +
                   "</rewrite></system.webServer></configuration>";
var reader = new StringReader(fullContents);

It can now be passed to AddIISUrlRewrite and we're good to go.

A mock HttpContext

The UrlRewriteModule reads from the HttpRequest instance and possibly writes to the HttpResponse instance. We need to mock both these. However the stuff we want to vary is all he URL data, so we pass that to a (fairly large) factory method:

private static HttpContext CreateHttpContext(Uri uri)
{
    var ctx = Mock.Of<HttpContext>();

    var resp = Mock.Of<HttpResponse>();
    var headers = new HeaderDictionary();
    Mock.Get(resp).Setup(x => x.Headers).Returns(headers);
    Mock.Get(ctx).Setup(x => x.Response).Returns(resp);

    var req = Mock.Of<HttpRequest>();
    Mock.Get(ctx).Setup(x => x.Request).Returns(req);
    Mock.Get(req).Setup(x => x.Path).Returns(new PathString(uri.AbsolutePath));
    Mock.Get(req).Setup(x => x.Scheme).Returns(uri.Scheme);
    Mock.Get(req).Setup(x => x.Host).Returns(new HostString(uri.Host, uri.Port));
    Mock.Get(req).Setup(x => x.PathBase).Returns(new PathString("/"));
    Mock.Get(req).Setup(x => x.QueryString).Returns(new QueryString(uri.Query));

    var variableFeature = Mock.Of<IServerVariablesFeature>();
    var features = new FeatureCollection();
    features.Set(variableFeature);
    Mock.Get(ctx).Setup(x => x.Features).Returns(features);

    Mock.Get(variableFeature).Setup(x => x["HTTP_HOST"]).Returns(uri.Host);
    Mock.Get(variableFeature).Setup(x => x["HTTP_URL"]).Returns(uri.AbsolutePath);
    Mock.Get(variableFeature).Setup(x => x["HTTPS"]).Returns(uri.Scheme == "https" ? "on" : "off");
    return ctx;
}

What to verify?

I was initially trying to verify the HttpResponse. We can do that and test the StatusCode and headers etc. However, I was Console.WriteLineing all the logging and it turns out the UrlRewrite middleware logs perfectly nice descriptions of what happens. It also (usually) stops at the rule that handles the URL, so if we take the last two outputs, we're quite good to go:

Request matched current UrlRewriteRule 'test'.
Location header '//tada' with status code '302'.

Request did not match current rule 'test'.
Current url is http://localhost:80/

If I could have my will, I'd like something like this, and I can have it. :)

http://localhost/abc => Request matched current UrlRewriteRule 'test'. Location header '//tada' with status code '302'.
http://localhost/ => Request did not match current rule 'test'. Current url is http://localhost:80/

So what loggerFactory sets up is an instance that keeps a local message log, and of course write to the console:

public class RedirectLogger : ILogger
{
    public List<string> Messages { get; } = new List<string>();

    public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter)
    {
        var output = formatter(state, exception);
        output = output.Replace("Request is done processing. ", "");
        output = output.Replace("Request is continuing in applying rules. ", "");
        Console.WriteLine(output);
        Messages.Add(output);
    }

    public bool IsEnabled(LogLevel logLevel)
    {
        return true;
    }

    public IDisposable BeginScope<TState>(TState state)
    {
        throw new NotImplementedException();
    }

    public override string ToString()
    {
        return String.Join(" ", Messages);
    }

    public string Last(int count = 2)
    {
        return String.Join(" ", Messages.TakeLast(count));
    }
}

I keep that instance around for the test to verify what was written to it.

Trying it together

The last piece here is Verify which we can use for larger "assertEquals" blocks.

I'd like one big report of different URLs I expect hit the server, and then the result for each url. So I set up a bunch of testcases like so:

public static Uri[] TestCases() => new[]
{
    new Uri("http://localhost/abc"),
    new Uri("https://localhost/abc"),
    new Uri("http://localhost/tadabcada"),
    new Uri("http://localhost/"),
    new Uri("http://localhost/tada")
};

Now we can loop over those and verify a report in our test:

[Test]
public async Task Handles_All_Rules()
{
    List<string> results = new List<string>();
    foreach(var uri in TestCases())
    {
        var ctx = CreateHttpContext(uri);

        redirectLogger.Messages.Clear();
        await middleware.Invoke(ctx);

        results.Add(uri + " => " + redirectLogger);
    }

    await Verify(String.Join(Environment.NewLine, results));
}

Our output becomes:

http://localhost/abc => Request matched current UrlRewriteRule 'test'. Location header '//tada' with status code '302'.
https://localhost/abc => Request matched current UrlRewriteRule 'test'. Location header '//tada' with status code '302'.
http://localhost/tadabcada => Request matched current UrlRewriteRule 'test'. Location header '//tada' with status code '302'.
http://localhost/ => Request did not match current rule 'test'. Current url is http://localhost:80/
http://localhost/tada => Request did not match current rule 'test'. Current url is http://localhost:80/tada

Fairly easy to read, and now we'll blow up if ever we break our rules.

I think the readability can be tweaked for even better readability and intuition, but for now I'm super happy I can test it. There are tons of other cases I haven't covered of course, but this is a very good starting place.

One quirk I've noticed is that the location header always gets a leading slash, but i think that's due to running in a test.

Hope you like it.
I'd love comments about usage or extensions. :)

All the code

Here's all the code reading the config from a resource file instead:

using Microsoft.AspNetCore.Rewrite;
using Moq;
using DiffEngine;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Features;
using Microsoft.Extensions.FileProviders;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Newtonsoft.Json.Linq;
using NUnit.Framework;

namespace UrlRewrite.Tests
{
    public class UrlRewriting
    {
        public static Uri[] TestCases() => new[]
        {
            new Uri("http://localhost/abc"),
            new Uri("https://localhost/abc"),
            new Uri("http://localhost/tadabcada"),
            new Uri("http://localhost/"),
            new Uri("http://localhost/tada")
        };

        [Test]
        public async Task Handles_All_Rules()
        {
            List<string> results = new List<string>();
            foreach(var uri in TestCases())
            {
                var ctx = CreateHttpContext(uri);

                redirectLogger.Messages.Clear();
                await middleware.Invoke(ctx);

                results.Add(uri + " => " + redirectLogger);
            }

            await Verify(String.Join(Environment.NewLine, results));
        }

        private RewriteMiddleware middleware = null!;
        private RedirectLogger redirectLogger = null!;

        [SetUp]
        public void Setup()
        {
            DiffTools.UseOrder(DiffTool.VisualStudio, DiffTool.Rider, DiffTool.VisualStudioCode);

            var options = CreateOptions(GetType().Assembly.GetManifestResourceStream("UrlRewrite.Tests.testroutes.xml")!);
            redirectLogger = new RedirectLogger();
            var loggerFactory = CreateLoggerFactory(redirectLogger);
            middleware = CreateMiddleware(loggerFactory, options);
        }

        private static HttpContext CreateHttpContext(Uri uri)
        {
            var ctx = Mock.Of<HttpContext>();

            var resp = Mock.Of<HttpResponse>();
            var headers = new HeaderDictionary();
            Mock.Get(resp).Setup(x => x.Headers).Returns(headers);
            Mock.Get(ctx).Setup(x => x.Response).Returns(resp);

            var req = Mock.Of<HttpRequest>();
            Mock.Get(ctx).Setup(x => x.Request).Returns(req);
            Mock.Get(req).Setup(x => x.Path).Returns(new PathString(uri.AbsolutePath));
            Mock.Get(req).Setup(x => x.Scheme).Returns(uri.Scheme);
            Mock.Get(req).Setup(x => x.Host).Returns(new HostString(uri.Host, uri.Port));
            Mock.Get(req).Setup(x => x.PathBase).Returns(new PathString("/"));
            Mock.Get(req).Setup(x => x.QueryString).Returns(new QueryString(uri.Query));

            var variableFeature = Mock.Of<IServerVariablesFeature>();
            var features = new FeatureCollection();
            features.Set(variableFeature);
            Mock.Get(ctx).Setup(x => x.Features).Returns(features);

            Mock.Get(variableFeature).Setup(x => x["HTTP_HOST"]).Returns(uri.Host);
            Mock.Get(variableFeature).Setup(x => x["HTTP_URL"]).Returns(uri.AbsolutePath);
            Mock.Get(variableFeature).Setup(x => x["HTTPS"]).Returns(uri.Scheme == "https" ? "on" : "off");
            return ctx;
        }

        private RewriteMiddleware CreateMiddleware(ILoggerFactory loggerFactory, RewriteOptions options)
        {
            return new RewriteMiddleware(_ => Task.CompletedTask, Mock.Of<IWebHostEnvironment>(), loggerFactory, new OptionsWrapper<RewriteOptions>(options));
        }

        private static ILoggerFactory CreateLoggerFactory(ILogger logger)
        {
            var loggerFactory = Mock.Of<ILoggerFactory>();
            Mock.Get(loggerFactory).Setup(x => x.CreateLogger(It.IsAny<string>())).Returns(logger);
            return loggerFactory;
        }

        private static RewriteOptions CreateOptions(Stream stream)
        {
            using var reader = new StreamReader(stream);
            var options = new RewriteOptions().AddIISUrlRewrite(reader);
            options.StaticFileProvider = Mock.Of<IFileProvider>();
            return options;
        }
    }

    public class RedirectLogger : ILogger
    {
        public List<string> Messages { get; } = new List<string>();

        public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter)
        {
            var output = formatter(state, exception);
            output = output.Replace("Request is done processing. ", "");
            output = output.Replace("Request is continuing in applying rules. ", "");
            Console.WriteLine(output);
            Messages.Add(output);
        }

        public bool IsEnabled(LogLevel logLevel)
        {
            return true;
        }

        public IDisposable BeginScope<TState>(TState state)
        {
            throw new NotImplementedException();
        }

        public override string ToString()
        {
            return String.Join(" ", Messages);
        }
    }
}

Lars-Erik's blog

Ramblings about Umbraco, .net and JavaScript development. With a sprinkle of other stuff.