Tag: testing

Categories
iOS Swift

Reading GraphQL queries from URLRequest in network tests

Back in 2019 I wrote about Testing networking code with custom URLProtocol on iOS. Recently, I was using the same approach for setting up network mocking for GraphQL requests, but then I stumbled on something unexpected. If we create an URLRequest which has httpBody set, then inside the custom URLProtocol implementation, httpBody method returns nil, and we need to access httpBodyStream instead. When dealing with GraphQL requests, the GraphQL query is part of the data set to httpBody. Reading the data from httpBody property would be straight-forward, then httpBodyStream on the other hand returns an InputStream and this requires a bit of code. InputStreams can only be read once, therefore while inspecting the URLRequest inside the URL protocol, we need to make sure not to read it twice. This is a nature of how input streams work.

In the snippet below, we can see a Data extension which adds a new initializer which takes in an InputStream. The implementation opens the stream, closes it when initializer is exited, reads data 512 bytes at the time. The read function of the InputStream returns number of read bytes if successful, negative value on error and 0 when buffer end has been reached which means that there is noting more to read.

extension Data {
init(inputStream: InputStream) throws {
inputStream.open()
defer { inputStream.close() }
self.init()
let bufferSize = 512
var readBuffer = [UInt8](repeating: 0, count: bufferSize)
while inputStream.hasBytesAvailable {
let readBytes = inputStream.read(&readBuffer, maxLength: bufferSize)
switch readBytes {
case 0:
break
case ..<0:
throw inputStream.streamError!
default:
append(readBuffer, count: readBytes)
}
}
}
}
view raw Data+InputStream.swift hosted with ❤ by GitHub

Let’s see how to use this extension and reading a GraphQL query from the URLRequest. In the example below we can see that the example GraphQL data is set to a dictionary with 3 keys: query, operationName, and variables. Therefore, we need to first turn the InputStream into Data and then decoding the data to a model type and after that reading the query. Since we know the query, we can proceed with writing network tests where the mocked response depends on the query.

// Example data set to URLRequest httpBody
// {
// "query": "query HeroNameAndFriends($episode: Episode) { hero(episode: $episode) { name friends { name } } }",
// "operationName": "",
// "variables": { "episode": "JEDI" }
// }
struct Payload: Decodable {
let query: String
}
let outputData = try Data(inputStream: httpBodyStream)
let payload = try JSONDecoder().decode(Payload.self, from: outputData)
print(payload.query)
// Prints: query HeroNameAndFriends($episode: Episode) { hero(episode: $episode) { name friends { name } } }
view raw Usage.swift hosted with ❤ by GitHub

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS Swift Swift Package

Basic unit-tests for SwiftUI view with ViewInspector

While using SwiftUI for building UIs, sooner or later we would like to write some unit-tests as well. Of course, we could always go for UI-tests, but these are much slower and therefore not so scalable if we would just want to have a basic verification for our view. It is easy to get going with writing unit-tests for UIKit code, but it is much more difficult for SwiftUI views. Currently, there seems to be two main ways: snapshot testing using the pointfreeco’s library or inspecting views with ViewInspector. Today, we are not going to compare these libraries and instead just take a look at how to get going with ViewInspector.

Although ViewInspector supports using bindings etc for updating the view while running the unit-test, I personally feel like these kinds of tests where we interact with the view is probably better for UI-tests. Therefore, in this blog post, we just take a look at a basic SwiftUI view and how to inspect it in a unit-tests.

Here we have a basic SwiftUI view which uses a view model to provide data for the view. The style property is driving the title of the view. A pretty simple example to demonstrate the usage of the ViewInspector library.

struct ContentView: View {
@StateObject var viewModel = ViewModel()
var body: some View {
VStack {
Image(systemName: "globe")
.imageScale(.large)
.foregroundColor(.accentColor)
Text(viewModel.title)
.font(.title)
}
.padding()
}
}
extension ContentView {
final class ViewModel: ObservableObject {
@Published var style: Style = .hello
var title: String {
switch style {
case .hello: return "Hello World!"
case .welcome: return "Welcome World!"
}
}
}
enum Style {
case hello, welcome
}
}
view raw ContentView.swift hosted with ❤ by GitHub

Time to add two unit-tests where we first configure the view model and then verify. It is also possible to write a test where we dynamically change the style property, but that requires some extra code for supporting it. The first step after adding the library and only adding to the unit-testing target is to opt-in the custom view for inspection. Without the ContentView extension the library is not able to inspect any views. In the example below, we are just using the find method for looking for a Text with specific string, which depends on the style property. I feel like this library is really nice for these kinds of unit-tests where we create a view and just verify what it is displaying.

@testable import SwiftUIExampleViewInspector
import SwiftUI
import ViewInspector
import XCTest
// Do not forget this!
extension ContentView: Inspectable {}
final class ContentViewTests: XCTestCase {
func testInitialTitle() throws {
let contentView = ContentView()
let text = try contentView.inspect()
.find(text: "Hello World!")
let font = try text.attributes().font()
XCTAssertEqual(font, Font.title)
}
func testWelcomeTitle() throws {
let viewModel = ContentView.ViewModel()
viewModel.style = .welcome
let contentView = ContentView(viewModel: viewModel)
_ = try contentView.inspect().find(text: "Welcome World!")
}
}
view raw ContentViewTests.swift hosted with ❤ by GitHub

In summary, I think that ViewInspector is a really nice library for unit-testing SwiftUI views. Since it requires more work to support reacting to view updates dynamically I feel like, at least for now, I am going to use it for static views and use UI-tests instead for tests simulating user interaction.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS Swift Swift Package

Handling never finishing async functions in Swift package tests

Why does my CI never finish and post a message to the merge request? Logged in to CI and oh, my merge job had been running for 23 minutes already, although typically it finishes in 4 minutes. What was going on? Nothing else than on unit-test marked with async was still waiting for an async function to finish. So what can we to avoid this? Let’s first create a Swift package which will be demonstrating the issue.

struct ImageLoader {
func loadImage(for identifier: String) async throws -> UIImage {
// Delay for 100 seconds
try await Task.sleep(nanoseconds: UInt64(100 * 1e9))
return UIImage()
}
}
view raw ImageLoader.swift hosted with ❤ by GitHub

And a simple unit-test for the successful case.

final class ImageLoaderTests: XCTestCase {
func testLoadingImageSuccessfully() async throws {
let imageLoader = ImageLoader()
_ = try await imageLoader.loadImage(for: "identifier")
}
}
view raw ImageLoaderTests.swift hosted with ❤ by GitHub

This test passes after 100 seconds, but clearly, we do not want to wait so long if something takes way too much time. Instead, we want to fail the test when it is still running after 5 seconds.

Exploring XCTestCase executionTimeAllowance

XCTestCase has a property called executionTimeAllowance what we can set. Ideally I would like to write something like executionTimeAllowance = 5 and Xcode would fail the test with a timeout failure after 5 seconds.

override func setUpWithError() throws {
executionTimeAllowance = 5 // gets rounded up to 60
}
view raw ExecutionTimeout.swift hosted with ❤ by GitHub

But if we read the documentation, then it mentions that the value set to this property is rounded up to the nearest minute value. In addition, this value is not used if you do not enable it explicitly: “To use this setting, enable timeouts in your test plan or set the -test-timeouts-enabled option to YES when using xcodebuild.”. If we are working on a Swift package, then I am actually not sure how to set it in the Package.swift so that it gets set when running the test from Xcode or from a command line.

Custom test execution with XCTestExpectation

One way to avoid never finishing tests is to use good old XCTestExpectation. We can set up a method which runs the async work and then waits for the test expectation with a timeout. If a timeout occurs, the test fails. If the async function throws an error, we can capture it, fail the test with XCTFail.

final class ImageLoaderTests: XCTestCase {
func testLoadingImageSuccessfully() {
execute(withTimeout: 5) {
let imageLoader = ImageLoader()
_ = try await imageLoader.loadImage(for: "identifier")
}
}
}
extension XCTestCase {
func execute(withTimeout timeout: TimeInterval, file: StaticString = #filePath, line: UInt = #line, workItem: @escaping () async throws -> Void) {
let expectation = expectation(description: "wait for async function")
var workItemError: Error?
let captureError = { workItemError = $0 }
let task = Task {
do {
try await workItem()
}
catch {
captureError(error)
}
expectation.fulfill()
}
waitForExpectations(timeout: timeout) { _ in
if let error = workItemError {
XCTFail("\(error)", file: file, line: line)
}
task.cancel()
}
}
}
view raw CustomAsyncHandler.swift hosted with ❤ by GitHub

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS Swift SwiftUI Xcode

Getting started with mocking networking in UI-tests on iOS

It is important to have stable unit-tests and UI-tests since no-one wants to encounter failures in tests which happen non-deterministically. Many of the iOS apps rely on networking, and therefore the content depends on what is fetched from servers. The last thing what we want to see is that an API outage or instability affects UI-tests. This time we’ll take a look at how to mock networking in UI-tests which differs from mocking networking in unit-tests since the whole app is going to be running as is and is controlled by a separate UI-testing runner app. A while ago I also covered unit-testing part in Testing networking code with custom URLProtocol on iOS, please take a look at that post for more information since we’ll be using the same custom URL protocol approach.

The main difference between unit-tests and UI-tests is that with unit-tests Xcode injects the unit-testing code into the app and then runs tests. UI-testing on the other hand rely on a separate test runner app which uses accessibility APIs for driving the user-interface in the app. This means that our network mocking code needs to be bundled with the app when we build it. The approach we are going to use is setting up a separate “UITestingSupport” Swift package, which is included only in debug builds. This library contains mocked data and configures the custom URL protocol and handles any network requests. Please see Linking a Swift package only in debug builds for more information on how to only link a package in debug builds.

I’ll be using a sample app “UITestingNetworking” for demonstrating how to set it up. The app has an app target and a local Swift package with a name “UITestingSupport”.

Xcode project layout with UITestingSupport package.

The first piece of the “UITestingSupport” package is a custom URLProtocol. All it does is providing a way to return either error or URLResponse and Data for a URLRequest. It is a simplified protocol. In an actual app we would want to control which requests are handled by it and which are not. Either because it is way too difficult to mock all the network request in all the tests at first, or we might also want to have some tests using an actual data coming from servers.

final class UITestingURLProcotol: URLProtocol {
override class func canInit(with request: URLRequest) -> Bool {
return true // TODO: only return true for requests we have mocked data
}
override class func canonicalRequest(for request: URLRequest) -> URLRequest {
return request
}
struct ResponseData {
let response: URLResponse
let data: Data
}
static var responseProvider: ((URLRequest) -> Result<ResponseData, Error>)?
override func startLoading() {
guard let client else { fatalError() }
if let responseProvider = Self.responseProvider {
switch responseProvider(request) {
case .success(let responseData):
client.urlProtocol(self, didReceive: responseData.response, cacheStoragePolicy: .notAllowed)
client.urlProtocol(self, didLoad: responseData.data)
client.urlProtocolDidFinishLoading(self)
case .failure(let error):
client.urlProtocol(self, didFailWithError: error)
client.urlProtocolDidFinishLoading(self)
}
}
else {
let error = NSError(domain: "UITestingURLProcotol", code: -1)
client.urlProtocol(self, didFailWithError: error)
}
}
override func stopLoading() {}
}
view raw UITestingURLProtocol.swift hosted with ❤ by GitHub

The second piece of the library is a UITestingNetworkHandler class which the app code will call, and it configures the custom URLProtocol and starts providing responses based on the “responseProvider” callback.

public final class UITestingNetworkHandler {
public static func register() {
URLProtocol.registerClass(UITestingURLProcotol.self)
UITestingURLProcotol.responseProvider = { request in
guard let url = request.url else { fatalError() }
switch (url.host, url.path) {
case ("augmentedcode.io", "/api/example"):
let response = HTTPURLResponse(url: url, statusCode: 200, httpVersion: nil, headerFields: nil)!
let data = "MyMockedData".data(using: .utf8)!
return .success(UITestingURLProcotol.ResponseData(response: response, data: data))
default:
fatalError("Unhandled")
}
}
}
}
view raw UITestingNetworkHandler.swift hosted with ❤ by GitHub
Simple example of providing response for a URLRequest.

The example above just handles one network request. For larger apps we probably want to have more component based implementation here since this file would otherwise grow a lot based on how many cases we want to handle. Another thing to note is that in some tests we want to mimic network request failures and in others successful requests but with different response data. This is not shown above, but can be implemented by providing the expected configuration flag through environment variables. XCUIApplication has a launchEnvironent property what we can set and then reading that value in the UITestingNetworkHandler with Process environment property. I’m thinking something like “MyAppAPIExampleResponseType” which equals to a number or some identifier.

The last piece is to call the register code when we are running the app in UI-tests.

@main
struct UITestingNetworkingApp: App {
var body: some Scene {
WindowGroup {
ContentView()
#if DEBUG
.onAppear(perform: {
guard CommandLine.arguments.contains("–uitesting") else { return }
UITestingNetworkHandler.register()
})
#endif
}
}
}
view raw App.swift hosted with ❤ by GitHub
Calling register only in debug builds and when a launch argument is set by the runner app.

And finally, an example UI-test which taps a button which in turn fetches some data from the network and then just displays the raw data in the app.

class UITestingNetworkingUITests: XCTestCase {
override func setUpWithError() throws {
continueAfterFailure = false
}
func testExample() throws {
let app = XCUIApplication()
app.launchArguments = ["–uitesting"]
app.launch()
app.buttons["Load Data"].tap()
XCTAssertEqual(app.staticTexts.element.label, "MyMockedData")
}
}
view raw UITestingNetworkingUITests.swift hosted with ❤ by GitHub
An example UI-test which sets launch argument which enables network mocking in the app.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee

Example Project

UITestingNetworking (GitHub)

Categories
Swift

Using XCTExpectFailure in XCTests

XCTExpectFailure is a function in XCTest framework which tackles the problem of managing failing tests. Test which are broken, but not ready to be fixed. An example use-case is when refactoring code which causes some tests to fail but since the whole refactoring is far for complete we might want to hold off fixing tests now because everything can change yet again. This is when we can use XCTExpectFailure function to mark the entire test or just a code block in a test as an expected failure. Every time we run tests, the test code runs but if it fails, the expected failure message is added to the test result. In addition, this failure does not fail the entire suit. This is important when there is a CI pipeline set up which requires that there are no failing tests. In reality the test is failing, but we have marked it to be OK for now. Something to keep in mind is that XCTest framework also has a XCTSkip functions. The main difference between XCTExpectFailure and XCTSkip is that the latter stops the test code execution immediately, where the former always runs the test. Therefore, XCTExpectFailure allows us to see when the test is passing again. Might be that after a lot of refactorings, the code is in a shape again where it produces expected output.

The simplest usage of XCTExpectFailure takes no arguments. If an error is thrown inside the test then the test is marked as an expected failure, but if the test does not throw any errors then XCTExpectFailure fails the test. This is due to the fact that XCTExpectFailure is strict by default strict. The strict mode means that a failure must be happening in the test or otherwise XCTExpectFailure fails the tests. When taking that refactoring example again, then that is useful for removing XCTExpectFailure calls as soon as the test starts passing. If non-strict mode is used, then the test is marked as passed if no failures are happening, although XCTExpectFailure is used within that test. An example use-case is that we’ll clean up all XCTExpectFailures at the end of the refactoring cycle when all the tests are passing again. In addition to the strict flag, there is also enabled flag which can be used for disabling the XCTExpectFailure based on environment or any other reason on run-time. Another thing what we can configure is what kind of failure is observed by passing in an issue matcher closure, where we can inspect the issue and decide if it should be considered as an expected failure or not in fine detail. An example could be that there is a very specific error thrown which we want to consider as expected failure, but all the other failures should lead to failing the test itself.

Different usages of the XCTExpectFailure are shown below.

XCTExpectFailure("Will be fixed in ticket XXX")
XCTExpectFailure("Will be fixed in ticket XXX", strict: false)
XCTExpectFailure("Will be fixed in ticket XXX", enabled: true, strict: false) {
    XCTAssertEqual(validateData(), false)
}
XCTExpectFailure("Will be fixed in ticket XXX", enabled: false, strict: false, failingBlock: {
    XCTAssertEqual(validateData(), false)
}, issueMatcher: { issue in
    issue.type == .assertionFailure
})

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS Swift

Code coverage for Swift Packages with Fastlane

Package.swift file in Swift packages describes the package contents including other dependencies, build targets and so on. Double-clicking a Package.swift file opens Xcode with that package and one can build and run tests. When adding continues integration to a Swift package then we would like to build, test, and also create a code coverage report. Fastlane is an excellent tool for such tasks and therefore let’s take a look on how to achieve that for packages with iOS target.

Code coverage support needs to be explicitly set and this can be done by using Swift Package Manager’s generate-xcodeproj command with having code coverage turned on. SPM then creates a Xcode project which has a slightly different scheme name with “-Package” suffix. The generated project file can then be used to run tests with Fastlane’s run_tests action. After running test we can use a tool like xcov for generating an HTML page with a code coverage report. One of the great features of the xcov is the ability to force a minimum code coverage percentage.

This approach works fine when we do not have any resource files in the package which are required for running tests. The problem is that the generate-xcodeproj command creates a project file without resource files. In those cases we would need to either use a separate standalone project or workspace file which has code coverage turned on.

Down below, we can see an example Fastfile which has a lane called “test_report” which generates a project file, runs tests and uses xcov for generating code coverage report. Additionally, it will delete the generated project file and any related folders.

require 'fileutils'
default_platform(:ios)
platform :ios do
desc "Run tests and create a unit-test report"
lane :test_report do
spm(
command: 'generate-xcodeproj',
enable_code_coverage: true
)
run_tests(
project: 'SPMPackage.xcodeproj',
scheme: 'SPMPackage-Package',
device: 'iPhone 12',
output_directory: './fastlane/UnitTests',
clean: true
)
xcov(
project: 'SPMPackage.xcodeproj',
scheme: 'SPMPackage-Package',
output_directory: './fastlane/CodeCoverage',
include_targets: 'SPMPackage.framework',
minimum_coverage_percentage: 80.0
)
if ENV['CI'] != 'true'
sh 'open ./UnitTests/report.html'
sh 'open ./CodeCoverage/index.html'
end
end
before_all do
FileUtils.remove_dir './CodeCoverage', true
FileUtils.remove_dir './UnitTests', true
end
after_all do
FileUtils.remove_dir './../SPMPackage.xcodeproj', true
end
end
view raw Fastfile.rb hosted with ❤ by GitHub

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
Foundation iOS macOS Swift Xcode

Creating a pre-push git hook in Swift script

Git hooks are scripts written in any scripting language and are triggered when important actions occur. Hooks are stored in the repository’s .git/hooks folder. The script needs to have an appropriate filename without a path extension and also have executable permissions. Push event is an excellent time for triggering unit-tests and making sure local changes have not broken any. Therefore we’ll look into how to create a pre-push script for an iOS project in Swift.

Quick introduction to scripts written in Swift

Setting up a simple script in Swift follows steps familiar from other languages. The script file needs to start with #!/usr/bin/swift followed by the actual script. A simple example script can look like this.

#!/usr/bin/swift
print("Example")
view raw Script.swift hosted with ❤ by GitHub
Example script written in Swift.

The command for running the script is swift Script.swift (if the filename is Script.swift). Another way is making the script executable by adding executable permissions to the file by running chmod +x Script.swift command. Then the script can be run with ./Script.swift (makes sense to drop the file extension).

Building a Xcode project for testing

The pre-push script contains of 3 steps: building the project for testing, running tests, and finally printing out the code coverage results. The first step catches build errors, the second step test failures, and the third step prints out code coverage results. Code coverage can be enabled in the scheme’s test action or adding -enableCodeCoverage YES to the xcodebuild command. Before we jump into creating a xcodebuild command with correct arguments then we’ll need to tackle the problem of calling the xcodebuild command line application from the Swift script. Command line applications can be invoked with the Foundation’s Process class. We’ll add an extension which deals with launching a specified command with zsh and printing out the standard output and error.

extension Process {
@discardableResult
static func runZshCommand(_ command: String) -> Int32 {
let process = Process()
process.launchPath = "/bin/zsh"
process.arguments = ["-c", command]
process.standardOutput = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.standardError = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.launch()
process.waitUntilExit()
(process.standardOutput as! Pipe).fileHandleForReading.readabilityHandler = nil
(process.standardError as! Pipe).fileHandleForReading.readabilityHandler = nil
return process.terminationStatus
}
}
view raw Process.swift hosted with ❤ by GitHub
Launching command line application with Process.

The next step in the script is to define the project related configuration and create the xcodebuild command. All the user defined arguments are wrapped in quotes for avoiding any issues with whitespaces. The command is pretty straight-forward. If there are any build errors then the result code is not equal to 0. Then we can use the same error code for exiting the Swift script with exit() function.

#!/usr/bin/swift
import Foundation
let projectType = "-workspace"
let projectPath = "SignalPath.xcworkspace"
let scheme = "SignalPathiOS"
let destinationDevice = "platform=iOS Simulator,name=iPhone 11 Pro Max"
let resultBundlePath = "PrePush.xcresult"
removeResultBundle(at: resultBundlePath)
print("Building for testing…")
let buildCommand = [
"xcodebuild",
"build-for-testing",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes
].joined(separator: " ")
let buildStatus = Process.runZshCommand(buildCommand)
if buildStatus != 0 {
exit(buildStatus)
}
extension String {
var wrappedInQuotes: String {
return "\"\(self)\""
}
}
view raw pre-push-1.swift hosted with ❤ by GitHub
Building a Xcode project for testing.

Running unit-tests

The command used for running unit-tests is fairly similar. Instead of build-without-testing we are using test-without-building argument and additionally provide a path where the result bundle is written to. This bundle contains information about the test run. Note that this path must not exist, otherwise xcodebuild stops with an error. Therefore we delete the existing file before running the command. Moreover, when there is a failure, we’ll clean up the path as well – pre-push script should not leave any temporary files.

removeResultBundle(at: resultBundlePath)
print("Running tests…")
let testCommand = [
"xcodebuild",
"test-without-building",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes,
"-resultBundlePath", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
let testStatus = Process.runZshCommand(testCommand)
if testStatus != 0 {
removeResultBundle(at: resultBundlePath)
exit(testStatus)
}
func removeResultBundle(at path: String) {
guard FileManager.default.fileExists(atPath: path) else { return }
try? FileManager.default.removeItem(atPath: path)
}
view raw pre-push-2.swift hosted with ❤ by GitHub
Running unit-tests in a pre-built project.

Printing out code coverage

Last step is optional but it is nice to see code coverage information when pushing changes to a server. Xcode provides a command line application for viewing coverage data in human readable form. One of the options is printing out code coverage per target which gives a nice and concise overview.

let coverageCommand = [
"xcrun",
"xccov",
"view",
"–only-targets",
"–report", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
Process.runZshCommand(coverageCommand)
removeResultBundle(at: resultBundlePath)
print("Success")
exit(0)
view raw pre-push-3.swift hosted with ❤ by GitHub
Printing out code coverage per target.

Summary

We looked into how to create a pre-push script in Swift. It called other command line applications for building the project, running the tests, and printing out code coverage information. The full script is available below, feel free to copy-paste it to your projects. The one last thing to consider is adding an alias in Terminal for easy installation: alias xcode_pre_push_add='cp ~/Dev/pre-push .git/hooks/pre-push && mate .git/hooks/pre-push' This just copies it from predefined location to the repository checkout and opens it in an editor for setting project related settings (replace mate with any editor).

#!/usr/bin/swift
import Foundation
let projectType = "-workspace"
let projectPath = "SignalPath.xcworkspace"
let scheme = "SignalPathiOS"
let destinationDevice = "platform=iOS Simulator,name=iPhone 11 Pro Max"
let resultBundlePath = "PrePush.xcresult"
print("Building for testing…")
let buildCommand = [
"xcodebuild",
"build-for-testing",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes
].joined(separator: " ")
let buildStatus = Process.runZshCommand(buildCommand)
if buildStatus != 0 {
exit(buildStatus)
}
removeResultBundle(at: resultBundlePath)
print("Running tests…")
let testCommand = [
"xcodebuild",
"test-without-building",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes,
"-resultBundlePath", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
let testStatus = Process.runZshCommand(testCommand)
if testStatus != 0 {
removeResultBundle(at: resultBundlePath)
exit(testStatus)
}
let coverageCommand = [
"xcrun",
"xccov",
"view",
"–only-targets",
"–report", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
Process.runZshCommand(coverageCommand)
removeResultBundle(at: resultBundlePath)
print("Success")
exit(0)
// MARK: –
extension String {
var wrappedInQuotes: String {
return "\"\(self)\""
}
}
extension Process {
@discardableResult
static func runZshCommand(_ command: String) -> Int32 {
let process = Process()
process.launchPath = "/bin/zsh"
process.arguments = ["-c", command]
process.standardOutput = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.standardError = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.launch()
process.waitUntilExit()
(process.standardOutput as! Pipe).fileHandleForReading.readabilityHandler = nil
(process.standardError as! Pipe).fileHandleForReading.readabilityHandler = nil
return process.terminationStatus
}
}
func removeResultBundle(at path: String) {
guard FileManager.default.fileExists(atPath: path) else { return }
try? FileManager.default.removeItem(atPath: path)
}
view raw pre-push.swift hosted with ❤ by GitHub
Full pre-push script for building, running, and printing code coverage.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS macOS Swift Xcode

Performance testing using XCTMetric

XCTMetric enables creating tests with measure blocks collecting information about CPU, memory and disk. In this post we’ll write UI-tests measuring a button tap what triggers writing to disk, allocating larger amount of memory and applying filters what requires CPU to do more work. It should be noted that XCTMetric can also be used in unit-tests.

Method under the test

The method we are going to write performance tests against is a simple method dealing with loading an image, writing data to disk, applying CIFilter and writing processed image to disk. In this example case, everything runs on a main thread what probably would not be a case in a real application.

@IBAction func process(_ sender: Any) {
	let image = UIImage(named: "Image")!
	imageStorage.store(image, filename: "original")
        
	let processedImage = ImageProcessor.processImage(image)
	imageStorage.store(image, filename: "processed")
	processedImageView.image = processedImage
}

XCTClockMetric for measuring taken time

XCTClockMetric is for measuring time taken by the block. Useful for catching regressions in longer running operations.

func testCalculateWithClockMetric() {
	let app = XCUIApplication()
	app.launch()
	measure(metrics: [XCTClockMetric()]) {
		app.buttons["Process"].tap()
	}
}

XCTCPUMetric for measuring CPU utilization

XCTCPUMetric measures CPU activity and output 3 different results: CPU time, CPU cycles and CPU instructions retired. CPUs have a feature called speculative execution what means that more instructions are completed than the actual program flow requires. Retired instructions are the instructions which were actually needed by the flow of the program. This feature speeds up the program execution as CPU can process data ahead of time. Example case would be if else where CPU processes both branches but only one branch is valid in the program flow.

func testCalculateWithCPUMetric() {
	let app = XCUIApplication()
	app.launch()
	measure(metrics: [XCTCPUMetric(application: app)]) {
		app.buttons["Process"].tap()
	}
}

XCTMemoryMetric for measuring allocated memory

XCTMemoryMetric measures allocated physical memory useful for testing operation allocating significant amount of memory (processing images).

func testCalculateWithMemoryMetric() {
	let app = XCUIApplication()
	app.launch()
	measure(metrics: [XCTMemoryMetric(application: app)]) {
		app.buttons["Process"].tap()
	}
}

XCTStorageMetric for measuring disk usage

XCTStorageMetric measures bytes written to the disk.

func testCalculateWithStorageMetric() {
	let app = XCUIApplication()
	app.launch()
	measure(metrics: [XCTStorageMetric(application: app)]) {
		app.buttons["Process"].tap()
	}
}

XCTOSSignpostMetric for measuring time between signposts

Apple provides signpost metric for application launch time for making it easy to add performance test measing launch time. In WWDC’19 session “Optimizing app launch” the suggested goal is 400 ms which is the duration of the app launch animation. XCTOSSignpostMetric has initializer for custom signpost as well.

func testLaunchPerformance() {
	if #available(macOS 10.15, iOS 13.0, tvOS 13.0, *) {
		measure(metrics: [XCTOSSignpostMetric.applicationLaunch]) {
			XCUIApplication().launch()
		}
	}
}

Summary

XCTMetric enables writing performance tests for performance critical parts of the application. We took a look at CPU, memory, storage and signpost metrics.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Example

MeasuringInTests (Xcode 11.3)

Categories
Generics iOS Swift UIKit

Testing networking code with custom URLProtocol on iOS

Testing networking code might sound tricky at first but in reality, it just means using custom URLProtocol what returns data we would like to. This allows testing the networking module without mocking URLSession. Using this approach we could do so much more, even integrating a third party networking library.

Networking class wrapping URLSession

Firstly, let’s set up a simple WebClient class what uses URLSession for initiating networking requests. It has a fetch method for loading URLRequest and transforming the response to expected payload type using Codable. As payload can be any type, we use generics here. Note that we need to pass in the payload type as a variable because we need the exact type when decoding the JSON data. How can we test this as URLSession would try to send an actual request to designated URL? As unit tests should behave exactly the same all the time and should not depend on external factors, then using a separate test server is not preferred. Instead, we can intercept the request and provide the response with custom URLProtocol.

final class WebClient {
private let urlSession: URLSession
init(urlSession: URLSession) {
self.urlSession = urlSession
}
func fetch<T: Decodable>(_ request: URLRequest, requestDataType: T.Type, completionHandler: @escaping (Result<T, FetchError>) -> Void) {
let dataTask = urlSession.dataTask(with: request) { (data, urlResponse, error) in
if let error = error {
DispatchQueue.main.async {
completionHandler(.failure(.connection(error)))
}
return
}
guard let urlResponse = urlResponse as? HTTPURLResponse else {
DispatchQueue.main.async {
completionHandler(.failure(.unknown))
}
return
}
switch urlResponse.statusCode {
case 200..<300:
do {
let payload = try JSONDecoder().decode(requestDataType, from: data ?? Data())
DispatchQueue.main.async {
completionHandler(.success(payload))
}
}
catch let jsonError {
DispatchQueue.main.async {
completionHandler(.failure(.invalidData(jsonError)))
}
}
default:
DispatchQueue.main.async {
completionHandler(.failure(.response(urlResponse.statusCode)))
}
}
}
dataTask.resume()
}
}
extension WebClient {
enum FetchError: Error {
case response(Int)
case invalidData(Error)
case connection(Error)
case unknown
}
}
view raw WebClient.swift hosted with ❤ by GitHub

Creating custom URLProtocol for unit tests

URLProtocol is meant to be overridden. Firstly, we’ll need to override canInit(with:) and return true here allowing URLSession to use this protocol for any URL request. Secondly, it is required to override canonicalRequest(for:) where we can just return the same request. Thirdly, startLoading, where we have the loading logic which uses class property for returning appropriate response. This allows us to set this property in unit tests and then returning the result when URLSession handles the fetch request. Finally, URLProtocol also needs to define stopLoading method what we can just leave empty as this protocol is not asynchronous.

final class TestURLProtocol: URLProtocol {
override class func canInit(with request: URLRequest) -> Bool {
return true
}
override class func canonicalRequest(for request: URLRequest) -> URLRequest {
return request
}
static var loadingHandler: ((URLRequest) -> (HTTPURLResponse, Data?, Error?))?
override func startLoading() {
guard let handler = TestURLProtocol.loadingHandler else {
XCTFail("Loading handler is not set.")
return
}
let (response, data, error) = handler(request)
if let data = data {
client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
client?.urlProtocol(self, didLoad: data)
client?.urlProtocolDidFinishLoading(self)
}
else {
client?.urlProtocol(self, didFailWithError: error!)
}
}
override func stopLoading() {}
}
view raw TestURLProtocol.swift hosted with ❤ by GitHub

Using TestURLProtocol for mocking network requests in unit tests

Setting up a unit test requires to set the TestURLProtocol’s loadingHandler and returning the data we would like to. Then we create URLSessionConfiguration and set our TestURLProtocol to protocolClasses. After that we can use this configuration for initialising URLSession and using this session in our WebClient which handles fetch requests. That is pretty much all we need to do for testing networking requests.

final class WebClientTests: XCTestCase {
override func tearDown() {
TestURLProtocol.loadingHandler = nil
}
struct TestPayload: Codable, Equatable {
let country: String
}
func testFetchingDataSuccessfully() {
let expected = TestPayload(country: "Estonia")
let request = URLRequest(url: URL(string: "https://www.example.com")!)
let responseJSONData = try! JSONEncoder().encode(expected)
TestURLProtocol.loadingHandler = { request in
let response = HTTPURLResponse(url: request.url!, statusCode: 200, httpVersion: nil, headerFields: nil)!
return (response, responseJSONData, nil)
}
let expectation = XCTestExpectation(description: "Loading")
let configuration = URLSessionConfiguration.ephemeral
configuration.protocolClasses = [TestURLProtocol.self]
let client = WebClient(urlSession: URLSession(configuration: configuration))
client.fetch(request, requestDataType: TestPayload.self) { (result) in
switch result {
case .failure(let error):
XCTFail("Request was not successful: \(error.localizedDescription)")
case .success(let payload):
XCTAssertEqual(payload, expected)
}
expectation.fulfill()
}
wait(for: [expectation], timeout: 1)
}
}
view raw WebClientTests.swift hosted with ❤ by GitHub

Summary

Testing networking code at first might sound daunting. But actually it just boils down to using custom URLProtocol and providing response we need to in our test.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Example project

TestingNetworkRequests (Xcode 5.0, Xcode 10.2.1)

Resources