Status Quo of Umbraco Testing 2017

It's been a year since giving my presentation about Unit Testing Umbraco at Codegarden 2016. Testing Umbraco is ever more popular, and I'm happy to see there's more people talking about it on twitter, at festivals and at gatherings. Recently, Christian Thillemann decided he was tired of waiting for the request I created last year about having Umbraco.Tests on Nuget. He went ahead and set up an AppVeyor script and we now have automatic releases of Our.Umbraco.Community.Tests on nuget whenever there's a new Umbraco version out. #H5YR, @kedde! :)

Of course, having this requirement and having to lean that heavily on an external assembly just to get to annoying internals in our core product is just an intermediate challenge. Come Umbraco 9 on .net Core, we'll (hopefully) be able to just spin up a complete hosted Umbraco site from our tests and inject / swap out whatever we'd fancy in that startup. Java and other platforms has had that option for years, so it's about time the .net community gets there. But enough ranting about that, let's just keep contributing so we get closer to running on .net Core.

Meanwhile...

In the mean time, I've been exploring further practices with automated testing. I'll get back to those in future posts and presentations. But while doing this, I stubled into an alternative practice to the one I've been preaching. It's mostly the same, but it challenges the inherital structure of Umbraco.Tests.

A really good principle when you write tests is letting your tests be interfaces to your code. Whenever you write a user interface, you take good care putting your logic in separate classes. The UI stuff is just UI stuff. Maybe our tests should be like that too? When you do that, you can even re-use your tests for different layers of your code! I'll not go into details about that in this post, but a natural result was to offload everything about Umbraco into its own class. I don't know why I didn't before, or why it took me so long to do it, but I suppose it's got something to do with closed mentality. When you see a pattern (inherited unit-tests), you automatically think it's a good one, and don't challenge it.

An epiphany

If you've gone through my post about the basics of testing Umbraco, you know you can inherit quite a few different base tests from Umbraco.Tests. They all have their uses, and I still recommend using the BaseDatabaseFactoryTest if you're testing stuff integrated with a database. However, most of what I write are different Surface-, Render- and *ApiControllers using fairly few stubbable things. All the front-end ones have the same basic initial state needs, and the backoffice ones just have fewer. So we end up inheriting BaseRoutingTest and calling GetRoutingContext(...) in all our setups. Being the smart devs you are, I'm sure a lot of you also ended up with some kind of MyUmbracoTestBase.

But there's something completely wrong with that! We're letting the dependency on Umbraco get in the way of our own automation code. We can't create a hirarchy of tests that isn't dependent on Umbraco. For instance, if we had a base class initializating our own domain in our core domain tests, we couldn't re-use that for our MVC tests. To do that we'd have to inherit our core base test class from Umbraco tests, and then Umbraco would leak into our core domain. We don't want that.

The solution

SRP your tests of course! Excercise the same dicipline by applying nice layering to your test automation code as you do to your other code. I ended up refactoring most of my test classes' setups and teardowns into a "Support" class with only three public methods. Here's how you'd set up a test for a simple SurfaceController with this method:

[TestFixture]
public class Adding_Numbers
{
    private UmbracoSupport support = new UmbracoSupport();

    [SetUp]
    public void Setup()
    {
        support.SetupUmbraco();
    }

    [TearDown]
    public void TearDown()
    {
        support.DisposeUmbraco();
    }

    [Test]
    public void Posting_AddModel_Calculates_Result()
    {
        const int expectedSum = 3;
        var controller = new SimpleSurfaceController();
        var result = (AdditionModel)controller.Add(1, 2);
        Assert.AreEqual(expectedSum, result.Model.Sum);
    }
}

I want to stress that this is not, and will not be a package. See below for reasons.

So it isn't that different from inheriting BaseRoutingTest. The only difference is that we delegate the setup and teardown to another type instance instead of delegating it to the base class. The UmbracoSupport class still derives from BaseRoutingTest, but it stops there. It won't leak further.

Note that NUnit creates an instance of your test fixture for each test, so you'll get a fresh UmbracoSupport instance for each of your tests.

The third method you'd want to call is PrepareController(). The moment you'd like to act upon the CurrentPage, or just use RedirectToCurrentUmbracoPage, you'll have to call support.PrepareController(controller). This wires up the Umbraco request context instance to your controller's state.

When you've done that, you've got quite a few properties on the support class that let's you get to the juicy stuff you'll want to stub:

The implementation

I've already implemented this class in the umbraco-unit-testing-samples repo. I'll post it here as well for you to copy uncritically. I won't go through the details. It's just a refactored version of what I've shown in previous posts.

using System;
using System.Globalization;
using System.Web;
using System.Web.Mvc;
using System.Web.Routing;
using Moq;
using Umbraco.Core;
using Umbraco.Core.Configuration.UmbracoSettings;
using Umbraco.Core.Models;
using Umbraco.Core.Persistence;
using Umbraco.Core.Services;
using Umbraco.Tests.TestHelpers;
using Umbraco.Web;
using Umbraco.Web.Mvc;
using Umbraco.Web.Routing;

namespace Umb.Testing.Tests.Support
{
    public class UmbracoSupport : BaseRoutingTest
    {
        public UmbracoContext UmbracoContext => umbracoContext;

        public new ServiceContext ServiceContext => serviceContext;

        public IPublishedContent CurrentPage => currentPage;

        public RouteData RouteData => routeData;

        public UmbracoHelper UmbracoHelper => umbracoHelper;

        public HttpContextBase HttpContext => umbracoContext.HttpContext;

        public string ContentCacheXml { get; set; }

        /// <summary>
        /// Initializes a stubbed Umbraco request context. Generally called from [SetUp] methods.
        /// Remember to call UmbracoSupport.DisposeUmbraco from your [TearDown].
        /// </summary>
        public void SetupUmbraco()
        {
            InitializeFixture();
            TryBaseInitialize();

            InitializeSettings();

            CreateCurrentPage();
            CreateRouteData();
            CreateContexts();
            CreateHelper();

            InitializePublishedContentRequest();
        }

        /// <summary>
        /// Cleans up the stubbed Umbraco request context. Generally called from [TearDown] methods.
        /// Must be called before another UmbracoSupport.SetupUmbraco.
        /// </summary>
        public void DisposeUmbraco()
        {
            TearDown();
        }

        /// <summary>
        /// Attaches the stubbed UmbracoContext et. al. to the Controller.
        /// </summary>
        /// <param name="controller"></param>
        public void PrepareController(Controller controller)
        {
            var controllerContext = new ControllerContext(HttpContext, RouteData, controller);
            controller.ControllerContext = controllerContext;

            routeData.Values.Add("controller", controller.GetType().Name.Replace("Controller", ""));
            routeData.Values.Add("action", "Dummy");
        }

        protected override string GetXmlContent(int templateId)
        {
            if (ContentCacheXml != null)
                return ContentCacheXml;

            return base.GetXmlContent(templateId);
        }

        private UmbracoContext umbracoContext;
        private ServiceContext serviceContext;
        private IUmbracoSettingsSection settings;
        private RoutingContext routingContext;
        private IPublishedContent currentPage;
        private RouteData routeData;
        private UmbracoHelper umbracoHelper;
        private PublishedContentRequest publishedContentRequest;

        protected override ApplicationContext CreateApplicationContext()
        {
            // Overrides the base CreateApplicationContext to inject a completely stubbed servicecontext
            serviceContext = MockHelper.GetMockedServiceContext();
            var appContext = new ApplicationContext(
                new DatabaseContext(Mock.Of<IDatabaseFactory>(), Logger, SqlSyntax, GetDbProviderName()),
                serviceContext,
                CacheHelper,
                ProfilingLogger);
            return appContext;
        }

        private void TryBaseInitialize()
        {
            // Delegates to Umbraco.Tests initialization. Gives a nice hint about disposing the support class for each test.
            try
            {
                Initialize();
            }
            catch (InvalidOperationException ex)
            {
                if (ex.Message.StartsWith("Resolution is frozen"))
                    throw new Exception("Resolution is frozen. This is probably because UmbracoSupport.DisposeUmbraco wasn't called before another UmbracoSupport.SetupUmbraco call.");
            }
        }

        private void InitializeSettings()
        {
            // Stub up all the settings in Umbraco so we don't need a big app.config file.
            settings = SettingsForTests.GenerateMockSettings();
            SettingsForTests.ConfigureSettings(settings);
        }

        private void CreateCurrentPage()
        {
            // Stubs up the content used as current page in all contexts
            currentPage = Mock.Of<IPublishedContent>();
        }

        private void CreateRouteData()
        {
            // Route data is used in many of the contexts, and might need more data throughout your tests.
            routeData = new RouteData();
        }

        private void CreateContexts()
        {
            // Surface- and RenderMvcControllers need a routing context to fint the current content.
            // Umbraco.Tests creates one and whips up the UmbracoContext in the process.
            routingContext = GetRoutingContext("http://localhost", -1, routeData, true, settings);
            umbracoContext = routingContext.UmbracoContext;
        }

        private void CreateHelper()
        {
            umbracoHelper = new UmbracoHelper(umbracoContext, currentPage);
        }

        private void InitializePublishedContentRequest()
        {
            // Some deep core methods fetch the published content request from routedata
            // others access it through the context
            // in any case, this is the one telling everyone which content is the current content.

            publishedContentRequest = new PublishedContentRequest(new Uri("http://localhost"), routingContext, settings.WebRouting, s => new string[0])
            {
                PublishedContent = currentPage,
                Culture = CultureInfo.CurrentCulture
            };

            umbracoContext.PublishedContentRequest = publishedContentRequest;

            var routeDefinition = new RouteDefinition
            {
                PublishedContentRequest = publishedContentRequest
            };

            routeData.DataTokens.Add("umbraco-route-def", routeDefinition);
        }
    }
}

Why no package?

You might be wondering why I didn't package this up into a nice reusable nuget package for y'all to install. It's because THIS ISN'T IT! It isn't a silver bullet. It's boilerplate. You're bound to want to add your own varying states and preconditions to this setup. If I'd packaged it, we'd need an "OnThis", "OnThat" and "OnWhatever" for everything that happens so you could extend it. I've done several PRs to the core to have Umbraco.Tests usable as it is now, and I don't want to create another such layer.

You might for instance have to fiddle a bit with CreateApplicationContext(), and I'm quite sure that any sligtly advanced site will have to have tests setting up stuff in FreezeResolution(). That'd be solvable with inheritance, but here we go again... ;)

Next up

There's a few doors that open when you extract everything out of your tests. I'm dying to show you more, but I'll have to leave a lame "to-be-continued" and hope I don't mess it up. Happy refactoring, and do go get that nuget package from @kedde!