Outside-In TDD/Integration tests with Custom Web Application Factory in ASP.NET

May 29, 2022
Posted in Testing
May 29, 2022 Adrià Arquimbau

Outside-In TDD/Integration tests with Custom Web Application Factory in ASP.NET

I find it very interesting to write about this topic since I have verified that it is difficult to find specific documentation grouping all the points of how to write a good E2E test for APIs in .Net Core using Custom Web Application Factory and at the same time that it is an integration test.

Beyond Traditional TDD

Traditional test-driven development is usually done only at a unit test level, so we are creating objects and calling functions or methods. Could be called “middle-out TDD” because you start in the middle of your application building domain logic, and then you assemble your application features from that domain code.

There are a few limitations to relying exclusively on unit tests, however. End-to-end testing is another valuable kind of test, adding you to another level of testing a whole flow in your Application/API and such tests are now feasible for developers to write, especially on the API. But traditional TDD doesn’t provide any guidance on how to incorporate end-to-end tests into your TDD workflow, or when to write them, or how to write them.

Also, the Traditional TDD code is usually tested with its real dependencies, the other code works within production. This is helpful for test realism and can catch bugs with how modules integrate with one another. But it means that making a change to one lower-level module can cause test failures in many higher-level modules. There can also be a problem with defect localization: the tests aren’t able to pinpoint where the bug is in a lower-level module, because they only see the problem that comes out the other end of the higher-level module.

Finally, building from the middle out can result in building functionality that is unused or difficult to use. Say you put a lot of effort into the Traditional TDD layer, a well-designed module for handling data, then afterwards try to integrate it with the rest of your application. Maybe it turns out your app gets all the data it needs from elsewhere and doesn’t actually need that module you put so much effort into. Maybe you discover that the interface of that module isn’t in the form that the rest of the app needs, and you need to rework it. Middle-out TDD can be vulnerable to wasted effort.

The Two-Level TDD Loop (Outside-in)

Outside-in TDD addresses all of these problems by adding end-to-end tests that are written as part of a second loop:

  • Red: write an E2E test and watch it fail
  • Green: to make it pass, step down to a unit test and use a Red-Green-Refactor cycle to implement only enough functionality to get past the current E2E test failure
  • Step back up and rerun the E2E test to see if it passes. If it fails with another failure, step back down to the unit test level again

These steps can be visualized as a two-level loop:

This style of TDD is called outside-in because you start from the outside of your application (the E2E test), and you step inward to implement only the functionality that’s needed from the outside.

Until when creating E2E Tests?

This decision should be an agreement by each team or company. As the E2E test doesn’t have the responsibility to test de logic (unit Tests responsibility), we could talk about working with one “Happy Path” E2E test for each controller, being this test is the most complete case in the controller and at the same time, we are testing all the integration and connections.

Real Code configuration

Create Client

The first step is to use and configure : IClassFixture<CustomWebApplicationFactory<Startup>> in our, Test Class

Doc: https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests

Our Custom E2E Test Class

public class ControllerTestClassShould : IClassFixture<CustomWebApplicationFactory>
    private readonly CustomWebApplicationFactory _factory;
    private readonly HttpClient _httpClient;
    public ControllerTestClassShould(CustomWebApplicationFactory factory)
        _factory = factory;
        _httpClient = _factory.CreateClient();

If we have our main configuration for our Database Context in the Startup we should have directly the configuration ready to work.

Inserting data (ARRANGE)

Insert data is important for tests where we need to retrieve or handle data. We could insert data using the Entities inside the context as well. I’m using this custom Scope configuration where I’m using a hardcoded DbContext service we can use the same service during the test execution.
In our test it would look like this:

await _factory.ExecuteDbContextAsync(async context =>
    await context.Entity.Add(newEntity);
    await context.SaveChangesAsync();

And in the configuration like this:

private async Task ExecuteScopeAsync(Func<IServiceProvider, Task> action)
    using var scope = Services.GetService<IServiceScopeFactory>().CreateScope();
    await action(scope.ServiceProvider);
public async Task ExecuteDbContext(Func<EquipmentDomainDbContext, Task> action)
    await ExecuteScopeAsync(sp => action(sp.GetService<EquipmentDomainDbContext>()));

Now we can create our previous environment inserting or modifying the needed data for our test using our DB context.

Calling the endpoint (ACT)

The act on our Outside-In test would be the endpoint call, we only have to specify the exact call using the required HTTP call with our factory field.

var response = new HttpResponseMessage(HttpStatusCode.NotImplemented);
var jsonRequest = JsonSerializer.Serialize(requestObject), ApiTestsHelper.JsonSerializerOptions);
var requestContent = new StringContent(jsonRequest, Encoding.UTF8, "application/json");
response = await _httpClient.PostAsync($"/url", requestContent);

Validations (ASSERT)

The most important part of the test is the validation of the response and the containing information, in this case, we must validate the response of the endpoint and if it contains data, and finally if we made changes to the database to check that these changes persist in it.

  • Response status
  • Response content
var json = await response.Content.ReadAsStringAsync();
var result = JsonSerializer.Deserialize<MyResponseDTO>(json);
  • If we want to check the information inside de DB we could retrieve it and check the expected changes using the same method that we used before to insert data.
await _factory.ExecuteDbContextAsync(async context =>
    var project = await context.Entity
      .Include(p => p.Entity2)
      .SingleOrDefaultAsync(x => x.Id == id);


Thank you for reading and feel free to comment and suggest changes. 🙂

, , , , , , , , , , , , , , , , , ,

Leave a Reply

Your email address will not be published.


I welcome you to contact me for more information.