Working on the bleeding edge is painful. Very painful at times. Throwing away our Team City builds because TFS 2010 didn’t support it really hurt. It was a big woops that I regret.

Moving to Visual Studio 2010 Beta was painful too. Visual Studio worked fine, but no ReSharper made it painful. Even in the early days, ReSharper was not stable in 2010, but after a few months, things look good now. ReSharper and VS 2010 are playing nice together.

There is one last pain point. No Xunit.net test runner. Xunitcontrib provides a ReSharper 5.0 build, but I was never able to get it to work…until now. Previously I was getting the same problem seen by others.

The plugin xunit could not be loaded from "C:\program files(x86)\jetBrains\resharper\v5.0\bin\plugins\xunit.dll" or one of its dependencies.  Operation is not supported.  (Exception from HRRESULT: 0×801311515)

I tried the best I could to right-click | Properties | Unblock all the files in the plug-in, but the “Unblock” never worked. I would get the same error. Going back into the properties dialog, the unblock would appear again. It looked like the unblock wasn’t sticking.

After a quick Google search, I found an alternative way to unblock files. Copy the files to a FAT 32 and back. The block is stripped. Sweet! My USB key is FAT 32, and after a quick copy and back. No more errors! ReSharper sees and runs my Xunit.net tests.

Just a quick overview of how I installed the plug-in:

1. Close Visual Studio.
2. Copy the contents of the ExternalAnnotations to:
   C:\Program Files (x86)\JetBrains\ReSharper\v5.0\Bin\ExternalAnnotations
3. Create a folder:
   C:\Program Files (x86)\JetBrains\ReSharper\v5.0\Bin\plugins\xunitcontrib.runner.resharper.5.0
4. Copy everything else to the USB key.
5. Copy it to the folder I just created in step #3.
6. Open Visual Studio and run all my tests.

We’re using the Data Annotations Validators in ASP.Net MVC 2.0, but there wasn’t a really great way to unit test them. I’ve read Brad Wilson’s blog post about how to set them up and unit test them, but I didn’t like how you relied on the name of the property as a string.

// Arrange
var propertyInfo = typeof(Contact).GetProperty("FirstName");

It would be nicer if that used lambda expressions to get the name of the property, so I set out to do just that. Before we begin, I need to remind you, as always, I don’t know what I’m doing. This is my first tiny venture into expression trees so it is possible isn’t the best way to do things, but so far, it’s working for me.

I created a base Specification to wrap up a lot of the messy bits of getting property names by lambdas. I took a look at how the ASP.Net MVC helpers did it and roughly followed their pattern.

public class DataAnnotationSpecification<T> : Specification
{

    public PropertyInfo Property(Expression<Func<T,object>> op)
    {
        return Property(GetInputName(op));
    }

    public PropertyInfo Property(string propertyName)
    {
        return typeof (T).GetProperty(propertyName);
    }

    public static string GetInputName<TProperty>(Expression<Func<T, TProperty>> expression)
    {
        // not sure I totally understand this
        if (expression.Body.NodeType == ExpressionType.Convert)
        {
            var ue = (UnaryExpression)expression.Body;
            return GetInputName((MemberExpression) ue.Operand );
        }

        if (expression.Body.NodeType == ExpressionType.MemberAccess)
        {
            return GetInputName((MemberExpression)expression.Body);
        }
        throw new NotImplementedException();
    }

    private static string GetInputName(MemberExpression memberExpression)
    {
        return memberExpression.Member.Name;
    }

}

I know this probably doesn’t cover an exhaustive list of expressions, but we’re only talking about Property access tests here so we’re probably in the clear.

To make things a bit nicer, I also created some extensions off PropertyInfo to help setup their assertions.

public static class AttributeAssertions
{

    public static void ShouldNotBeRequired(this PropertyInfo @this)
    {
        @this.ShouldNotHaveAttribute<RequiredAttribute>();
    }

    public static void ShouldBeRequired(this PropertyInfo @this)
    {
        @this.ShouldHaveAttribute<RequiredAttribute>();
    }

    private static T PropertyAttributeOn<T>(ICustomAttributeProvider propertyInfo)
    {
        return propertyInfo.GetCustomAttributes(typeof (T), false)
            .Cast<T>()
            .FirstOrDefault();
    }

    public static void ShouldHaveAttribute<T>(this PropertyInfo @this)
    {
        PropertyAttributeOn<T>(@this).ShouldNotBeNull();
    }
    public static void ShouldNotHaveAttribute<T>(this PropertyInfo @this)
    {
        PropertyAttributeOn<T>(@this).ShouldBeNull();
    }
}

We only have the ShouldBeRequired and ShouldNotBeRequired right now, but you could see it will be easy to extend upon them.  The finished spec looks something like this.

public class When_testing_the_state_of_the_CustomerViewData
    : DataAnnotationSpecification<CustomerViewData>
{
    [Then] public void the_customer_first_name_is_required()
    {
        Property(x => x.CustomerFirstName).ShouldBeRequired();
    }

    [Then] public void the_customer_middle_name_is_not_required()
    {
        Property(x => x.CustomerMiddleName).ShouldNotBeRequired();
    }

    [Then] public void the_customer_last_name_is_required()
    {
        Property(x=>x.CustomerLastName).ShouldBeRequired();
    }

    [Then] public void the_email_address_is_required()
    {
        Property(x=>x.CustomerEmailAddress).ShouldBeRequired();
    }

}

That’s not so bad, and it stands up to any refactoring we might do.

We’re just starting down the path of serious and extensive testing with mocks and fakes. Our initial tests were all done using Rhino Mocks, mostly because it had a well established reputation and we had no reason not to give it a try. The syntax is beautiful and did everything we needed, until we tried to make a test similar to this one pass successfully.

public interface IFoo
{
    string Bar();
}

[Fact] public void TestWithRhino()
{
    var mock = MockRepository.GenerateStub<IFoo>();
    mock.Stub(x => x.Bar()).Return("bar");
    mock.Stub(x => x.Bar()).Return("baz");
    Assert.Equal("baz",mock.Bar());
}

We couldn’t, at least we couldn’t using strictly stubs. From what I could gather, we could if we used a more complicated for of mocking besides stubs. We only need stubs, or fakes, because we don’t need to test all the interaction, just the ending state.

Since we don’t have any serious stock in using Rhino, as an alternative I decided to take a look at using Moq. The syntax for Moq is not as nice. It is more wordy, but I do like how it doesn’t rely on extensions methods for everything. But what about the sample test?

[Fact] public void TestWithMoq()
{
    var mock = new Mock<IFoo>();
    mock.Setup(x => x.Bar()).Returns("bar");
    mock.Setup(x => x.Bar()).Returns("baz");
    Assert.Equal("baz", mock.Object.Bar());
}

It passes! Switching wasn’t that difficult, but where is the ugly syntax I‘ve run into so far? Well this in Rhino:

MockOrdersRepository.AssertWasNotCalled(x=>x.StartOrder(null,null,null,null));

…becomes this in Moq:

MockOrdersRepository   .Verify(x=>x.StartOrder( It.IsAny<string>(),                            It.IsAny<string>(),                            It.IsAny<string>(),                            It.IsAny<string>()),              Times.Never());

It might be I am not familiar enough with either to say that is the equivalent assertion of both, but it is working for us. I am probably missing something that is causing Moq to look more verbose.

As a follow up to my previous post, testing when ModelState is relatively simple, but leaves me feelings a little uneasy about my tests.

Here is an example tests, stripped down to remove some of the unnecessary bits

public class When_submitting_lead_data_that_is_in_an_invalid_state : Specification
{
    // class fields...etc

    protected override void Given()
    {
        base.Given();
        _mockOrdersRepository = MockRepository.GenerateStub<IOrdersRepository>();
        _checkoutController = new CheckoutController(_mockOrdersRepository);

        _leadDataWithEmptyPlanName = new PreCheckoutViewData();

        // add an error to the model state
        _checkoutController.ModelState.AddModelError("", "");

        _exceptionWasThrown = false;
    }

    protected override void When()
    {
        base.When();
        try
        {
            _checkoutController.Checkout(_leadDataWithEmptyPlanName);
        } catch(ArgumentNullException)
        {
            _exceptionWasThrown = true;
        }
    }

    [Then] public void an_exception_is_thrown()
    {
        _exceptionWasThrown.ShouldBe(true);
    }
}

This feels pretty loose to me, but there can be another tests that tests which states the model will appear invalid.

We recently pushed a feature into production that had a pretty serious flaw. If gone unfound it could have caused headaches with billing the wrong customer the wrong amount and all kinds of other potentially damning effects. Luckily for us, we also had a second bug. The feature that was to send the notification e-mail that data was ready to be corrupted by our bug was misconfigured when it moved to the production environment. Lucky for us, no one was being told they should go execute our buggy code, so no one did and we found it, fixed it and all those actions items that queued up ran through without a hitch.

Now lucky for us or not, we did fail to find two serious bugs. It highlights the need for thorough acceptance testing. Thorough being the key word there.

The first bug existed if action items queued up and were acted upon in a certain order. Testing was done in this area, but the items never fell in the order where we found the bug. Would acceptance tests go so far to say:

• Act on the first item in the list and verify the actions were taken correctly.
• Act on the second item in the list…
• Act on a list with a single item…
• Act on a list with two items…
• Act on a list with n items…

How detailed do you get? 

The first bug was a pretty lame mistake, although I feel SQL should have been alerted when we attempted to do an udate  a single from a statement that returned multiples. I don’t know how it makes sense to choose the first row and go about your business. 

As for the second, testing the production configuration is hard. We do have some junk data laying around in the production DB that we can manipulate, and probably should have to verify the e-mails were sending. Config changes between the two are difficult to manage. Keeping the e-mail address, servers, roles, etc in sync to the correct environment is made simpler by branches, but verification isn’t.

In reference to my previous post about collectively running unit tests, I thought I would offer this ReSharper tip.

ReSharper’s unit test runner is just plain awesome. Why MSTests don’t work like that by default astounds me…and is enough reason for me not to use them (although there are several more). One option you have for aggregating which tests you want to run in a single session is to right click on the folder that contains their source, right click and either Run or Debug them.

I’m looking for a way to run unit tests in all open documents, but haven’t found that one yet.

I’ve been spending the better part of two days doing this same process over and over. It doesn’t feel like I’m testing anything of value.

I am not doing it in the C# 3.0ish way, but it is better to explain it that way. I have a class and a façade (MyFacade) that I want to extend and test.

class Foo {
     public string Name { get; set; }
}

class MyFacade {
     public IMyDependency Depend { get; set; }
}

interface IMyDependency {
     IList<Foo> GetAllFoos();
}

So I write my test first like a good boy person should. I know my façade should give me back a list of the Name properties from the Foo objects it knows about.

[TestFixture]
public class MyFacadeTests {
    [Test]
    public void GetFooNames_ReturnsOnlyName() {
        FakeDependancy fake = new FakeDependency();
        MyFacade m = new MyFacade() { Depend = fake };
        Assert.That(m.GetFooNames(), Is.EquivalentTo(fake.FakeFoos.Select(x => x.Name)));
    }
}

I need that FakeDependency, so I go ahead and create that. This would be a like my repository that will allow me to initialize the fake data.
class FakeDependency : IMyDependency {

    // My local fake repository
    public IList<Foo> FakeFoos = new List<Foo>{
                new Foo{ Name="A"},
                new Foo{ Name="B"},
                new Foo{ Name="B"}};

    public IList<Foo> GetAllFoos() {
        return FakeFoos;
    }
}

OK. So now that I have that, I add my GetFooNames stub to MyFacade.

class MyFacade {
    public IMyDependency Depend { get; set; }

    public IList<string> GetFooNames()
    {
        return null;
    }
}

Run my test. It fails. Good. Now to implement that method….uhh I just copy my assert statement:

fake.FakeFoos.Select(x => x.Name)

…replace “fake.FakeFoos” with “Depend.GetAllFoos()” and I get:

class MyFacade {
    public IMyDependency Depend { get; set; }

    public IEnumerable<string> GetFooNames() {
        return Depend.GetAllFoos().Select(x => x.Name);
    }
}

Build it. Run it. It passes. What did I really test since I just copied the assert statement into the implementation body and changed the source? Did I test anything? That my copy code does the same thing in two places? Am I wasting time?
I rolled it around with a friend and decided to take out the select in the assert statement and replace it with a declarative list so it would look like this:

Assert.That(m.GetFooNames(), Is.EquivalentTo(new[]{"A","B","C"}));

That make me feel a little less “copy and pastey”, but now I have to know my fake is setup to give me three letters for the names. Let’s add a little bit to the fake to make it easier to configure. I am hesitant to do this since I don’t want to add code to my test classes, but I think in the end it is more readable than a more raw configuration.
So I added this to my test class:

public FakeDependency With(int i)
{
    FakeFoos.Clear();
    for (int j = 0; j < i; j++)
    {
        FakeFoos.Add(new Foo());
    }
    return this;
}

public FakeDependency Named(string name)
{
    for (int i = 0; i < FakeFoos.Count; i++)
    {
        // We want them to be 1 based and not zero based
        FakeFoos[i].Name = string.Format("{0} {1}", name, i + 1);
    }
    return this;
}

And use it in my test like this:

[Test]
public void GetFooNames_ReturnsOnlyName() {
    FakeDependency fake = new FakeDependency().With(3).Named("Fake");
    MyFacade m = new MyFacade() { Depend = fake };
    Assert.That(m.GetFooNames(), Is.EquivalentTo(new[]{"Fake 1","Fake 2","Fake 3"}));
}

I’m a lot happier with that than copying the same code from the test to the implementation classes.

Recently I’ve been able to extend my unit test code coverage as I’ve been introducing more features into our code base. I know this is something that should always be done, but for whatever reason, that’s not the case. What I’ve come to notice is our current method of organizing our unit test makes adding new features and test for new features awkward.

Currently our unit tests align perfectly with the class they test. So for example, if we have a class:

namespace MyCompany.Domain{
     public class Foo{}
}

We have, or should have, a test fixture in a separate assembly that looks like this:

using MyCompany.Domain;
using NUnit.Framework;
namespace MyCompany.Domain.Tests{
    [TestFixture]
    public class FooTests{}
}

This is probably a pretty typical strategy for organization. It is intuitive where to find tests for a type. Were I’m loosing my taste for this method is as the solution grows to several assemblies or layers, adding new unit tests for a specific feature becomes a scattered mess of changes. If I need to add a new field, I add new tests in the UI.Tests, the Domain.Tests, Repository.Tests, Core.Tests, etc.  Makes sense though right? You have to pass that new field through all the layers. I’m ok with the number of tests I need to write, but I don’t like having to go to eight different places to run them when I want to test my feature. It would be nice to align them them to the feature, or tag them in such a way I could find them easily. It could be that we are over layered and that is the real problem. We could be an onion.

Shrek: Example? Okay, er… ogres… are… like onions. 
Donkey: [sniffs onion] They stink? 
Shrek: Yes…NO! 
Donkey: Or they make you cry. 
Shrek: No! 
Donkey: Oh, you leave them out in the sun and they turn brown and start sproutin’ little white hairs.

We are definitely an onion….or an ogre, but that’s a whole different analogy.

There is a nice article in CoDe Magazine by Scott Bellware that talks about BDD. Most of it I didn’t see as valuable to my every day activities until he started talking about SpecUnit .Net. It was, in theory, what I’ve been looking for. Align my tests with my features/defects/requirements / whatever. SpecUnit .Net does some nice human readable report generation that I don’t need, but why would I need a new tool when I could just categorize my tests in folders that line up with my real world problems.

That gets me close, but there is no way to run the entire category without using the command line. Still doable, but not as nice as ReSharper’s unit test runner. It would be nice to get a Run all tests in this category from the context menu, but I’m not sure if that would be easy to do across the different frameworks. Again it would be nice to save my unit test session so I don’t loose it every time I close the IDE.

While we’re dreaming, Maybe we could invert the relationship and make the class under test and attribute and the tests themselves could align directly with the requirements. Something like this for example:

[Test][TypeUnderTest(typeof(Foo))] public void Correctly_Calculates_Bar(){
    //...
}

Or we could even get fancy with generics:

[Test<Foo>]
public void Correctly_Calculates_Bar(){
    //...
}

The IDE, ReSharper or someone could let me run all the tests for Foo if I wanted. 

After Josh’s comments I thought I would add a couple screen shots of using the Category attribute to group the tests as needed. You can apply the Category attribute to the method or to the TestFixture.

    [TestFixture]
    public class MyClassTests {

        [Category("Add the ability to do work")]
        [Test] public void DoWork_WithNoArgs_DoesWork(){  }

        [Category("Add the ability to do additional work ")]
        [Category("Add the ability to do work")]
        [Test]
        public void DoWork_WithAllArgs_DoesWork() {   }
    }

You can select Categories from the unit test runner’s Group by: combo box.

It then will group your tests by their category. I did find the UI a bit buggy. At first it wasn’t applying the grouping, but after a couple attempts they started to show up.

Thanks again Josh. That will keep me content for the time being. Now if I could only inspect the entire solution for tests in the category I would really happy.