XCTExpectFailure is a function in XCTest framework which tackles the problem of managing failing tests. Test which are broken, but not ready to be fixed. An example use-case is when refactoring code which causes some tests to fail but since the whole refactoring is far for complete we might want to hold off fixing tests now because everything can change yet again. This is when we can use XCTExpectFailure function to mark the entire test or just a code block in a test as an expected failure. Every time we run tests, the test code runs but if it fails, the expected failure message is added to the test result. In addition, this failure does not fail the entire suit. This is important when there is a CI pipeline set up which requires that there are no failing tests. In reality the test is failing, but we have marked it to be OK for now. Something to keep in mind is that XCTest framework also has a XCTSkip functions. The main difference between XCTExpectFailure and XCTSkip is that the latter stops the test code execution immediately, where the former always runs the test. Therefore, XCTExpectFailure allows us to see when the test is passing again. Might be that after a lot of refactorings, the code is in a shape again where it produces expected output.
The simplest usage of XCTExpectFailure takes no arguments. If an error is thrown inside the test then the test is marked as an expected failure, but if the test does not throw any errors then XCTExpectFailure fails the test. This is due to the fact that XCTExpectFailure is strict by default strict. The strict mode means that a failure must be happening in the test or otherwise XCTExpectFailure fails the tests. When taking that refactoring example again, then that is useful for removing XCTExpectFailure calls as soon as the test starts passing. If non-strict mode is used, then the test is marked as passed if no failures are happening, although XCTExpectFailure is used within that test. An example use-case is that we’ll clean up all XCTExpectFailures at the end of the refactoring cycle when all the tests are passing again. In addition to the strict flag, there is also enabled flag which can be used for disabling the XCTExpectFailure based on environment or any other reason on run-time. Another thing what we can configure is what kind of failure is observed by passing in an issue matcher closure, where we can inspect the issue and decide if it should be considered as an expected failure or not in fine detail. An example could be that there is a very specific error thrown which we want to consider as expected failure, but all the other failures should lead to failing the test itself.
Different usages of the XCTExpectFailure are shown below.
XCTExpectFailure("Will be fixed in ticket XXX")
XCTExpectFailure("Will be fixed in ticket XXX", strict: false)
XCTExpectFailure("Will be fixed in ticket XXX", enabled: true, strict: false) {
XCTAssertEqual(validateData(), false)
}
XCTExpectFailure("Will be fixed in ticket XXX", enabled: false, strict: false, failingBlock: {
XCTAssertEqual(validateData(), false)
}, issueMatcher: { issue in
issue.type == .assertionFailure
})
If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.
Augmented Code 