Requiring min code coverage in Swift packages with GitHub actions

I have written before about running tests and getting code coverage data for Swift packages in Running tests in Swift package with GitHub actions and Code coverage for Swift Packages with Fastlane. In this post, I am going to revisit the problem and use only tools from Xcode: xcodebuild and xccov. One downside of the approach in Code coverage for Swift Packages with Fastlane is that it required to generate Xcode project which is a deprecated feature in Xcode 12.5, and also it used xcov for forcing code coverage which does not seem to be maintained. Therefore, take two is here. I have a tiny Swift package on GitHub called IndexedDataStore, and we’ll add min code coverage requirement with GitHub action to that package.

Generating xcresult bundle with code coverage information

IndexedDataStore Swift package already has a GitHub action set up which runs tests for macOS and iOS platforms. As we’ll need to run several commands, then we’ll add a new ci-ios-code-coverage.sh script to Scripts folder and then the GitHub action will just invoke that script. This allows to run the same commands easily in the local development environment. Let’s take a look at the first half of the script, which builds and runs tests and generates a result bundle with code coverage information.

#!/bin/sh
SCHEME="IndexedDataStore"
RESULT_BUNDLE="CodeCoverage.xcresult"
RESULT_JSON="CodeCoverage.json"
MIN_CODE_COVERAGE=50.0 # should really be higher =)
# Pre-clean
if [ -d $RESULT_BUNDLE ]; then
rm -rf $RESULT_BUNDLE
fi
if [ -f $RESULT_JSON ]; then
rm $RESULT_JSON
fi
# Build
set -o pipefail && env NSUnbufferedIO=YES xcodebuild build-for-testing -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES | xcpretty
# Test
set -o pipefail && env NSUnbufferedIO=YES xcodebuild test-without-building -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES -resultBundlePath $RESULT_BUNDLE | xcpretty
Building and running tests with code coverage enabled.

The pre-clean section of the script makes sure that we do not have files present. The RESULT_BUNDLE is created by xcodebuild and RESULT_JSON later when processing the bundle. Although we could skip deleting the RESULT_JSON then on the other hand RESULT_BUNDLE must always be deleted or otherwise xcodebuild will log a warning and wouldn’t create a new one. The run tests section is separated into two steps: build the package for testing and then running tests without building. An alternative would be to use test argument instead of build-for-testing and test-without-building. The current setup makes it easier to see which step failed: was it a build error or a test error. Code coverage is enabled by passing enableCodeCoverage=YES and specifying a path where the result bundle should be created with resultBundlePath. Other things to note is that we are using set -o pipefail for making sure the script wouldn’t ignore any errors and would fail immediately. Also, we’ll pipe xcodebuild’s output to xcpretty for a bit nicer output in logs. If we run this piece of script then we end up with a CodeCoverage.xcbundle which contains code coverage information in the package root.

Extracting line coverage and forcing min coverage

For extracting code coverage information from the CodeCoverage.xcresult bundle we can use Apple’s xccov command line tool which was built exactly for that. The first step is that we’ll convert CodeCoverage.xcresult to CodeCoverage.json file with xccov.

CodeCoverage.json

The second step is to parse the create json file which lists all the targets and their code coverage. I am not fully sure why, but it contains two targets with the same name IndexedDataStore. One target contains correct code coverage, but the other one is just an empty target but with the same name. Therefore we’ll need to filter out the empty one which luckily is not so difficult with jq command line tool. The jq command first takes an array for key targets and then finds a target which has a name “IndexedDataStore” and value for key executableLines is greater than 0. After that, we can access the key lineCoverage in the found target. Finally, we’ll convert the code coverage to percentages, but as the value is in float then we’ll need to use the bc command.

set -o pipefail && env NSUnbufferedIO=YES xcrun xccov view –report –json $RESULT_BUNDLE > $RESULT_JSON
CODE_COVERAGE=$(cat $RESULT_JSON | jq '.targets[] | select( .name == "IndexedDataStore" and .executableLines > 0 ) | .lineCoverage')
CODE_COVERAGE=$(echo $CODE_COVERAGE*100.0 | bc)
Extracting line coverage from the xcresult.

After extracting the code coverage, we can proceed with forcing a minimum code coverage. Yet again, we’ll use bc command when comparing two float values and then checking if the comparison was true. If code coverage is too low, we’ll print red error message and in case it was enough, we’ll print success message in green colour.

COVERAGE_PASSES=$(echo "$CODE_COVERAGE > $MIN_CODE_COVERAGE" | bc)
if [ $COVERAGE_PASSES -ne 1 ]; then
printf "\033[0;31mCode coverage %.1f%% is less than required %.1f%%\033[0m\n" $CODE_COVERAGE $MIN_CODE_COVERAGE
exit -1
else
printf "\033[0;32mCode coverage is %.1f%%\033[0m\n" $CODE_COVERAGE
fi
Printing code coverage result.
Success message when min code coverage is 50%.
Error message when min code coverage requirement is 90%.

Calling the script from GitHub action

The created script can now be called from a GitHub action.

name: CI
on:
push:
branches: [ main ]
pull_request:
branches:
'*'
jobs:
tests:
runs-on: macos-latest
steps:
uses: actions/checkout@v2
name: Lint
run: swiftlint
name: Build for macOS
run: swift build -v
name: Run macOS tests
run: swift test -v
name: Run iOS tests
run: sh ./Scripts/ci-ios-code-coverage.sh
view raw ci.yml hosted with ❤ by GitHub
GitHub action which calls the created script.
GitHub action log which ran the script.

Summary

In this blog post we revisited code coverage set up for Swift packages. By using xcodebuild and xccov we created code coverage setup for Swift packages.

If this was helpful, please let me know on Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s