Tag: code coverage

Categories
iOS Swift Swift Package

Requiring min code coverage in Swift packages with GitHub actions

I have written before about running tests and getting code coverage data for Swift packages in Running tests in Swift package with GitHub actions and Code coverage for Swift Packages with Fastlane. In this post, I am going to revisit the problem and use only tools from Xcode: xcodebuild and xccov. One downside of the approach in Code coverage for Swift Packages with Fastlane is that it required to generate Xcode project which is a deprecated feature in Xcode 12.5, and also it used xcov for forcing code coverage which does not seem to be maintained. Therefore, take two is here. I have a tiny Swift package on GitHub called IndexedDataStore, and we’ll add min code coverage requirement with GitHub action to that package.

Generating xcresult bundle with code coverage information

IndexedDataStore Swift package already has a GitHub action set up which runs tests for macOS and iOS platforms. As we’ll need to run several commands, then we’ll add a new ci-ios-code-coverage.sh script to Scripts folder and then the GitHub action will just invoke that script. This allows to run the same commands easily in the local development environment. Let’s take a look at the first half of the script, which builds and runs tests and generates a result bundle with code coverage information.

#!/bin/sh
SCHEME="IndexedDataStore"
RESULT_BUNDLE="CodeCoverage.xcresult"
RESULT_JSON="CodeCoverage.json"
MIN_CODE_COVERAGE=50.0 # should really be higher =)
# Pre-clean
if [ -d $RESULT_BUNDLE ]; then
rm -rf $RESULT_BUNDLE
fi
if [ -f $RESULT_JSON ]; then
rm $RESULT_JSON
fi
# Build
set -o pipefail && env NSUnbufferedIO=YES xcodebuild build-for-testing -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES | xcpretty
# Test
set -o pipefail && env NSUnbufferedIO=YES xcodebuild test-without-building -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES -resultBundlePath $RESULT_BUNDLE | xcpretty
view raw ci-ios-code-coverage.sh hosted with ❤ by GitHub
Building and running tests with code coverage enabled.

The pre-clean section of the script makes sure that we do not have files present. The RESULT_BUNDLE is created by xcodebuild and RESULT_JSON later when processing the bundle. Although we could skip deleting the RESULT_JSON then on the other hand RESULT_BUNDLE must always be deleted or otherwise xcodebuild will log a warning and wouldn’t create a new one. The run tests section is separated into two steps: build the package for testing and then running tests without building. An alternative would be to use test argument instead of build-for-testing and test-without-building. The current setup makes it easier to see which step failed: was it a build error or a test error. Code coverage is enabled by passing enableCodeCoverage=YES and specifying a path where the result bundle should be created with resultBundlePath. Other things to note is that we are using set -o pipefail for making sure the script wouldn’t ignore any errors and would fail immediately. Also, we’ll pipe xcodebuild’s output to xcpretty for a bit nicer output in logs. If we run this piece of script then we end up with a CodeCoverage.xcbundle which contains code coverage information in the package root.

Extracting line coverage and forcing min coverage

For extracting code coverage information from the CodeCoverage.xcresult bundle we can use Apple’s xccov command line tool which was built exactly for that. The first step is that we’ll convert CodeCoverage.xcresult to CodeCoverage.json file with xccov.

CodeCoverage.json

The second step is to parse the create json file which lists all the targets and their code coverage. I am not fully sure why, but it contains two targets with the same name IndexedDataStore. One target contains correct code coverage, but the other one is just an empty target but with the same name. Therefore we’ll need to filter out the empty one which luckily is not so difficult with jq command line tool. The jq command first takes an array for key targets and then finds a target which has a name “IndexedDataStore” and value for key executableLines is greater than 0. After that, we can access the key lineCoverage in the found target. Finally, we’ll convert the code coverage to percentages, but as the value is in float then we’ll need to use the bc command.

set -o pipefail && env NSUnbufferedIO=YES xcrun xccov view –report –json $RESULT_BUNDLE > $RESULT_JSON
CODE_COVERAGE=$(cat $RESULT_JSON | jq '.targets[] | select( .name == "IndexedDataStore" and .executableLines > 0 ) | .lineCoverage')
CODE_COVERAGE=$(echo $CODE_COVERAGE*100.0 | bc)
view raw ci-ios-code-coverage-2.sh hosted with ❤ by GitHub
Extracting line coverage from the xcresult.

After extracting the code coverage, we can proceed with forcing a minimum code coverage. Yet again, we’ll use bc command when comparing two float values and then checking if the comparison was true. If code coverage is too low, we’ll print red error message and in case it was enough, we’ll print success message in green colour.

COVERAGE_PASSES=$(echo "$CODE_COVERAGE > $MIN_CODE_COVERAGE" | bc)
if [ $COVERAGE_PASSES -ne 1 ]; then
printf "\033[0;31mCode coverage %.1f%% is less than required %.1f%%\033[0m\n" $CODE_COVERAGE $MIN_CODE_COVERAGE
exit -1
else
printf "\033[0;32mCode coverage is %.1f%%\033[0m\n" $CODE_COVERAGE
fi
view raw ci-ios-code-coverage-3.sh hosted with ❤ by GitHub
Printing code coverage result.
Success message when min code coverage is 50%.
Error message when min code coverage requirement is 90%.

Calling the script from GitHub action

The created script can now be called from a GitHub action.

name: CI
on:
push:
branches: [ main ]
pull_request:
branches:
– '*'
jobs:
tests:
runs-on: macos-latest
steps:
– uses: actions/checkout@v2
– name: Lint
run: swiftlint
– name: Build for macOS
run: swift build -v
– name: Run macOS tests
run: swift test -v
– name: Run iOS tests
run: sh ./Scripts/ci-ios-code-coverage.sh
view raw ci.yml hosted with ❤ by GitHub
GitHub action which calls the created script.
GitHub action log which ran the script.

Summary

In this blog post we revisited code coverage set up for Swift packages. By using xcodebuild and xccov we created code coverage setup for Swift packages.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
iOS Swift

Code coverage for Swift Packages with Fastlane

Package.swift file in Swift packages describes the package contents including other dependencies, build targets and so on. Double-clicking a Package.swift file opens Xcode with that package and one can build and run tests. When adding continues integration to a Swift package then we would like to build, test, and also create a code coverage report. Fastlane is an excellent tool for such tasks and therefore let’s take a look on how to achieve that for packages with iOS target.

Code coverage support needs to be explicitly set and this can be done by using Swift Package Manager’s generate-xcodeproj command with having code coverage turned on. SPM then creates a Xcode project which has a slightly different scheme name with “-Package” suffix. The generated project file can then be used to run tests with Fastlane’s run_tests action. After running test we can use a tool like xcov for generating an HTML page with a code coverage report. One of the great features of the xcov is the ability to force a minimum code coverage percentage.

This approach works fine when we do not have any resource files in the package which are required for running tests. The problem is that the generate-xcodeproj command creates a project file without resource files. In those cases we would need to either use a separate standalone project or workspace file which has code coverage turned on.

Down below, we can see an example Fastfile which has a lane called “test_report” which generates a project file, runs tests and uses xcov for generating code coverage report. Additionally, it will delete the generated project file and any related folders.

require 'fileutils'
default_platform(:ios)
platform :ios do
desc "Run tests and create a unit-test report"
lane :test_report do
spm(
command: 'generate-xcodeproj',
enable_code_coverage: true
)
run_tests(
project: 'SPMPackage.xcodeproj',
scheme: 'SPMPackage-Package',
device: 'iPhone 12',
output_directory: './fastlane/UnitTests',
clean: true
)
xcov(
project: 'SPMPackage.xcodeproj',
scheme: 'SPMPackage-Package',
output_directory: './fastlane/CodeCoverage',
include_targets: 'SPMPackage.framework',
minimum_coverage_percentage: 80.0
)
if ENV['CI'] != 'true'
sh 'open ./UnitTests/report.html'
sh 'open ./CodeCoverage/index.html'
end
end
before_all do
FileUtils.remove_dir './CodeCoverage', true
FileUtils.remove_dir './UnitTests', true
end
after_all do
FileUtils.remove_dir './../SPMPackage.xcodeproj', true
end
end
view raw Fastfile.rb hosted with ❤ by GitHub

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee
Categories
Foundation iOS macOS Swift Xcode

Creating a pre-push git hook in Swift script

Git hooks are scripts written in any scripting language and are triggered when important actions occur. Hooks are stored in the repository’s .git/hooks folder. The script needs to have an appropriate filename without a path extension and also have executable permissions. Push event is an excellent time for triggering unit-tests and making sure local changes have not broken any. Therefore we’ll look into how to create a pre-push script for an iOS project in Swift.

Quick introduction to scripts written in Swift

Setting up a simple script in Swift follows steps familiar from other languages. The script file needs to start with #!/usr/bin/swift followed by the actual script. A simple example script can look like this.

#!/usr/bin/swift
print("Example")
view raw Script.swift hosted with ❤ by GitHub
Example script written in Swift.

The command for running the script is swift Script.swift (if the filename is Script.swift). Another way is making the script executable by adding executable permissions to the file by running chmod +x Script.swift command. Then the script can be run with ./Script.swift (makes sense to drop the file extension).

Building a Xcode project for testing

The pre-push script contains of 3 steps: building the project for testing, running tests, and finally printing out the code coverage results. The first step catches build errors, the second step test failures, and the third step prints out code coverage results. Code coverage can be enabled in the scheme’s test action or adding -enableCodeCoverage YES to the xcodebuild command. Before we jump into creating a xcodebuild command with correct arguments then we’ll need to tackle the problem of calling the xcodebuild command line application from the Swift script. Command line applications can be invoked with the Foundation’s Process class. We’ll add an extension which deals with launching a specified command with zsh and printing out the standard output and error.

extension Process {
@discardableResult
static func runZshCommand(_ command: String) -> Int32 {
let process = Process()
process.launchPath = "/bin/zsh"
process.arguments = ["-c", command]
process.standardOutput = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.standardError = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.launch()
process.waitUntilExit()
(process.standardOutput as! Pipe).fileHandleForReading.readabilityHandler = nil
(process.standardError as! Pipe).fileHandleForReading.readabilityHandler = nil
return process.terminationStatus
}
}
view raw Process.swift hosted with ❤ by GitHub
Launching command line application with Process.

The next step in the script is to define the project related configuration and create the xcodebuild command. All the user defined arguments are wrapped in quotes for avoiding any issues with whitespaces. The command is pretty straight-forward. If there are any build errors then the result code is not equal to 0. Then we can use the same error code for exiting the Swift script with exit() function.

#!/usr/bin/swift
import Foundation
let projectType = "-workspace"
let projectPath = "SignalPath.xcworkspace"
let scheme = "SignalPathiOS"
let destinationDevice = "platform=iOS Simulator,name=iPhone 11 Pro Max"
let resultBundlePath = "PrePush.xcresult"
removeResultBundle(at: resultBundlePath)
print("Building for testing…")
let buildCommand = [
"xcodebuild",
"build-for-testing",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes
].joined(separator: " ")
let buildStatus = Process.runZshCommand(buildCommand)
if buildStatus != 0 {
exit(buildStatus)
}
extension String {
var wrappedInQuotes: String {
return "\"\(self)\""
}
}
view raw pre-push-1.swift hosted with ❤ by GitHub
Building a Xcode project for testing.

Running unit-tests

The command used for running unit-tests is fairly similar. Instead of build-without-testing we are using test-without-building argument and additionally provide a path where the result bundle is written to. This bundle contains information about the test run. Note that this path must not exist, otherwise xcodebuild stops with an error. Therefore we delete the existing file before running the command. Moreover, when there is a failure, we’ll clean up the path as well – pre-push script should not leave any temporary files.

removeResultBundle(at: resultBundlePath)
print("Running tests…")
let testCommand = [
"xcodebuild",
"test-without-building",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes,
"-resultBundlePath", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
let testStatus = Process.runZshCommand(testCommand)
if testStatus != 0 {
removeResultBundle(at: resultBundlePath)
exit(testStatus)
}
func removeResultBundle(at path: String) {
guard FileManager.default.fileExists(atPath: path) else { return }
try? FileManager.default.removeItem(atPath: path)
}
view raw pre-push-2.swift hosted with ❤ by GitHub
Running unit-tests in a pre-built project.

Printing out code coverage

Last step is optional but it is nice to see code coverage information when pushing changes to a server. Xcode provides a command line application for viewing coverage data in human readable form. One of the options is printing out code coverage per target which gives a nice and concise overview.

let coverageCommand = [
"xcrun",
"xccov",
"view",
"–only-targets",
"–report", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
Process.runZshCommand(coverageCommand)
removeResultBundle(at: resultBundlePath)
print("Success")
exit(0)
view raw pre-push-3.swift hosted with ❤ by GitHub
Printing out code coverage per target.

Summary

We looked into how to create a pre-push script in Swift. It called other command line applications for building the project, running the tests, and printing out code coverage information. The full script is available below, feel free to copy-paste it to your projects. The one last thing to consider is adding an alias in Terminal for easy installation: alias xcode_pre_push_add='cp ~/Dev/pre-push .git/hooks/pre-push && mate .git/hooks/pre-push' This just copies it from predefined location to the repository checkout and opens it in an editor for setting project related settings (replace mate with any editor).

#!/usr/bin/swift
import Foundation
let projectType = "-workspace"
let projectPath = "SignalPath.xcworkspace"
let scheme = "SignalPathiOS"
let destinationDevice = "platform=iOS Simulator,name=iPhone 11 Pro Max"
let resultBundlePath = "PrePush.xcresult"
print("Building for testing…")
let buildCommand = [
"xcodebuild",
"build-for-testing",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes
].joined(separator: " ")
let buildStatus = Process.runZshCommand(buildCommand)
if buildStatus != 0 {
exit(buildStatus)
}
removeResultBundle(at: resultBundlePath)
print("Running tests…")
let testCommand = [
"xcodebuild",
"test-without-building",
"-quiet",
projectType, projectPath.wrappedInQuotes,
"-scheme", scheme.wrappedInQuotes,
"-destination", destinationDevice.wrappedInQuotes,
"-resultBundlePath", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
let testStatus = Process.runZshCommand(testCommand)
if testStatus != 0 {
removeResultBundle(at: resultBundlePath)
exit(testStatus)
}
let coverageCommand = [
"xcrun",
"xccov",
"view",
"–only-targets",
"–report", resultBundlePath.wrappedInQuotes
].joined(separator: " ")
Process.runZshCommand(coverageCommand)
removeResultBundle(at: resultBundlePath)
print("Success")
exit(0)
// MARK: –
extension String {
var wrappedInQuotes: String {
return "\"\(self)\""
}
}
extension Process {
@discardableResult
static func runZshCommand(_ command: String) -> Int32 {
let process = Process()
process.launchPath = "/bin/zsh"
process.arguments = ["-c", command]
process.standardOutput = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.standardError = {
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { handler in
guard let string = String(data: handler.availableData, encoding: .utf8), !string.isEmpty else { return }
print(string)
}
return pipe
}()
process.launch()
process.waitUntilExit()
(process.standardOutput as! Pipe).fileHandleForReading.readabilityHandler = nil
(process.standardError as! Pipe).fileHandleForReading.readabilityHandler = nil
return process.terminationStatus
}
}
func removeResultBundle(at path: String) {
guard FileManager.default.fileExists(atPath: path) else { return }
try? FileManager.default.removeItem(atPath: path)
}
view raw pre-push.swift hosted with ❤ by GitHub
Full pre-push script for building, running, and printing code coverage.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Support me on Patreon
Donate with Paypal
Buy me a coffee