Categories
Swift

Static DocC documentation for Swift packages

Swift 5.6 released with Xcode 13.3 implements evolution proposals extensible built tools (SE-0303) and its extension command plugins (SE-0332). This opens up plugins for Swift packages. Along with that, Apple released DocC command plugin for Swift packages, which supports generating static webpages containing the documentation of the package.

Swift-DocC plugin

Apple’s Swift-DocC plugin comes with pretty rich documentation which covers many aspects of the documentation generation process. Something to keep in mind still is that the generated website can’t just be opened with Safari like we might have been used to when using Jazzy. The plugin has a separate preview command if we want to open the documentation locally. That command starts a local web server which renders the site.

Getting started with Swift-DocC plugin

As an example, we’ll take my IndexedDataStore Swift package and see what are the steps to generate and preview the documentation. But before that, for local usage, I would like to highlight the fact that Xcode’s Product menu contains a “Build Documentation” command which generates documentation and adds it to the Developer Documentation window.

Xcode product menu with build documentation menu item.
Xcode product menu with build documentation menu item.
Xcode documentation viewer with locally built documentation.
Xcode documentation viewer with locally built documentation.


OK, back to generating HTML webpages ourselves. The very first thing we need to do is adding the docc plugin as a dependency to our Swift package. If we have done that, then we have access to new commands which the plugin defines.

dependencies: [
  .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
],

Let’s take a look at the preview command at first which generates documentation, spins up a local web server which renders it.

swift package --disable-sandbox preview-documentation --target IndexedDataStore
Building for debugging...
Build complete! (0.13s)
Template: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/docc/render
========================================
Starting Local Preview Server
	 Address: http://localhost:8000/documentation/indexeddatastore
========================================
The generated documentation site.
The generated documentation site.

The other command is the one which generates the static documentation webpage, which we can then commit to GitHub and let the GitHub pages to render.

swift package \
    --allow-writing-to-directory ./docs \
    generate-documentation \
    --target IndexedDataStore \
    --disable-indexing \
    --output-path ./docs \
    --transform-for-static-hosting \
    --hosting-base-path IndexedDataStore

Since plugin commands run under a sandboxed environment, we’ll need to explicitly define which folder is writable with the --allow-writing-to-directory argument. The --disable-indexing argument disables generating index, which is used by Xcode or other IDEs. The --transform-for-static-hosting removes the need to have any routing rules on the web server. And finally, --hosting-base-path defines the base-path of the documentation. Meaning, if the GitHub repository name is IndexedDataStore then we should pass in IndexedDataStore. Otherwise, relative links in the generated webpage are incorrect. The full format of the URL when it is pushed to a branch and GitHub pages is configured to read from the pushed branch with relative path set to /docs is: https://<username>.github.io/<repository-name>/documentation/<target-name> .

GitHub pages configuration where the branch is set to docc-documentation and relative path to /docs.
GitHub pages configuration where the branch is set to docc-documentation and relative path to /docs.

For IndexedDataStore, it is https://laevandus.github.io/IndexedDataStore/documentation/indexeddatastore/. Note that the target name is in lowercase. The link won’t work since I switched GitHub pages back to the Jazzy documentation, which is auto-generated on merge.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Categories
Swift Swift Package

Creating a signed Swift package collection

Swift 5.5 brought us a new feature which allows creating Swift package collections (SE-0291). In this blog post we’ll go through required steps and create a package collection with Augmented Code packages. At the time of writing, there is only one package: IndexedDataStore.

Install Swift Package Collection Generator

First, we’ll need to install a tool which Apple created for package collections. It is called Swift Package Collection Generator. We’ll need to clone the repository, build it and then either using the built tools in their current location or also installing them to /usr/local/bin for easy access later on.


git clone https://github.com/apple/swift-package-collection-generator.git
cd swift-package-collection-generator
swift build --configuration release
install .build/release/package-collection-generate /usr/local/bin/package-collection-generate
install .build/release/package-collection-diff /usr/local/bin/package-collection-diff
install .build/release/package-collection-sign /usr/local/bin/package-collection-sign
install .build/release/package-collection-validate /usr/local/bin/package-collection-validate

Collection description in JSON

When the tool is installed, then the next step is to create a definition for the collection. Supported keys in that JSON file are available here: PackageCollectionsModel/Formats/v1.md.

{
  "name": "Augmented Code Collection",
  "overview": "Packages created by Augmented Code",
  "author": {
    "name": "Toomas Vahter"
  },
  "packages": [
    { "url": "https://github.com/laevandus/IndexedDataStore.git" }
  ]
}

Next step is to feed that JSON into the package-collection-generate tool which fetches additional metadata for these packages (–auth-token argument with GitHub personal access token must be used if the GitHub access is not already set up with SSH).

package-collection-generate input.json output.json --verbose --pretty-printed

The output.json looks like this:

{
  "formatVersion" : "1.0",
  "generatedAt" : "2022-01-08T07:05:44Z",
  "generatedBy" : {
    "name" : "Toomas Vahter"
  },
  "name" : "Augmented Code Collection",
  "overview" : "Packages created by Augmented Code",
  "packages" : [
    {
      "keywords" : [

      ],
      "license" : {
        "name" : "MIT",
        "url" : "https://raw.githubusercontent.com/laevandus/IndexedDataStore/main/LICENSE"
      },
      "readmeURL" : "https://raw.githubusercontent.com/laevandus/IndexedDataStore/main/README.md",
      "url" : "https://github.com/laevandus/IndexedDataStore.git",
      "versions" : [
        {
// … version descriptions which are pretty long

Signing the package collection

Before we go ahead and sign the package collection, we’ll need to prepare certificates. Open Keychain Access and then from the main menu Keychain Access > Certificate Assistant > Request Certificate from a Certificate Authority. Use your email and name and check the “Saved to disk” option. The next step is uploading the certificate request file to Apple. Uploading is done in Certificates, Identifiers & Profiles by tapping on the plus button and selecting Swift Package Collection Certificate. After clicking on the Continue button, we can upload the certificate request file we created with the Keychain Access. After that, we’ll download the certificate and double-clicking on the certificate file adds it to Keychain Access. Before we can sign the collection, the next step is exporting the private key from Keychain Access. Look for “Swift Package Collection” certificate, expand the item which reveals the private key, right-click on it and export it (set a password). Keychain Access creates .p12 file, which we’ll need to convert to .pem (set a password when asked). In the example below, I saved the exported private key to swift_package.p12.

openssl pkcs12 -nocerts -in swift_package.p12 -out swift_package.pem
openssl rsa -in swift_package.pem -out swift_package_rsa.pem

Now we have ready for signing the package collection as we have .cer and .pem files prepared.

package-collection-sign output.json output-signed.json swift_package_rsa.pem swift_package.cer

When the command is successful, we have an output-signed.json file, which we can share and add to Xcode.

Adding a new package collection

A new package collection can be added in Xcode by navigating to File > Add Packages sheet and clicking on the plus button and selecting Add Swift Package Collection. Xcode asks for a https URL of the collection. One option is to upload the signed collection json file to GitHub and then passing in an URL to the raw representation of the file. The URL to Augmented Code’s package collection is available here.

Package collection view in Xcode showing added Augmented Code Packages collection.
Package collection view in Xcode.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Categories
Swift

Adding async await to existing completion handler based code

Xcode 13 with Swift 5.5 toolchain brings async-await to the Swift language. The aim of this blog post is not to cover everything it brings, but instead concentrate on seeing how completion handler based code can be wrapped with async await. In the end, we have reused existing code but made it available to be called from async task contexts. For demonstrating this, we’ll take my IndexedDataStore Swift package and add async await supported methods to existing completion handler based API.

The IndexedDataStore package has async methods for loading data and storing data.

func loadData<T>(forIdentifier identifier: Identifier, dataTransformer: @escaping (Data) -> T?, completionHandler: @escaping (T?) -> Void)

func storeData(_ dataProvider: @escaping () -> Data?, identifier: Identifier = UUID().uuidString, completionHandler: @escaping (Result<Identifier, Error>) -> Void) {

We can easily add async await methods to these completion hander based methods. We’ll need to use one of the withCheckedXXXContinuation functions. This allows us to add a method to IndexedDataStore class like this:

public func loadData<T>(forIdentifier identifier: Identifier, dataTransformer: @escaping (Data) -> T?) async -> T? {
return await withCheckedContinuation({ continuation in
self.loadData(forIdentifier: identifier, dataTransformer: dataTransformer) { object in
continuation.resume(returning: object)
}
})
}

What happens here is that we use the async withCheckedContinuation function, which gives us a hook into async await machinery. The function suspends the current task in hand and calls the completion handler with a continuation object. Then we can use our completion handler based async code and resume the suspended task when we are done by calling resume.

When we are dealing with completion handlers which use the Result type or provide an error value, then we’ll need to use withCheckedThrowingContinuation instead. This function provides a continuation object which supports throwing errors. This is exactly the case with the storeData function.

public func storeData(_ dataProvider: @escaping () -> Data?, identifier: Identifier = UUID().uuidString) async throws -> Identifier {
return try await withCheckedThrowingContinuation({ continuation in
self.storeData(dataProvider, identifier: identifier) { result in
continuation.resume(with: result)
}
})
}

When it comes to unit-testing then async await makes the testing code so much shorter. We can annotate test functions with the async keyword and do not need to deal with XCTestExpectations.

func testStoreAndLoadAsync() async throws {
let identifier = try await dataStore.storeData({ "Data".data(using: .utf8) }, identifier: "abc")
XCTAssertEqual(identifier, "abc")
let string = await dataStore.loadData(forIdentifier: "abc", dataTransformer: { String(decoding: $0, as: UTF8.self) })
XCTAssertEqual(string, "Data")
}

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Categories
iOS Swift Swift Package

Requiring min code coverage in Swift packages with GitHub actions

I have written before about running tests and getting code coverage data for Swift packages in Running tests in Swift package with GitHub actions and Code coverage for Swift Packages with Fastlane. In this post, I am going to revisit the problem and use only tools from Xcode: xcodebuild and xccov. One downside of the approach in Code coverage for Swift Packages with Fastlane is that it required to generate Xcode project which is a deprecated feature in Xcode 12.5, and also it used xcov for forcing code coverage which does not seem to be maintained. Therefore, take two is here. I have a tiny Swift package on GitHub called IndexedDataStore, and we’ll add min code coverage requirement with GitHub action to that package.

Generating xcresult bundle with code coverage information

IndexedDataStore Swift package already has a GitHub action set up which runs tests for macOS and iOS platforms. As we’ll need to run several commands, then we’ll add a new ci-ios-code-coverage.sh script to Scripts folder and then the GitHub action will just invoke that script. This allows to run the same commands easily in the local development environment. Let’s take a look at the first half of the script, which builds and runs tests and generates a result bundle with code coverage information.

#!/bin/sh
SCHEME="IndexedDataStore"
RESULT_BUNDLE="CodeCoverage.xcresult"
RESULT_JSON="CodeCoverage.json"
MIN_CODE_COVERAGE=50.0 # should really be higher =)
# Pre-clean
if [ -d $RESULT_BUNDLE ]; then
rm -rf $RESULT_BUNDLE
fi
if [ -f $RESULT_JSON ]; then
rm $RESULT_JSON
fi
# Build
set -o pipefail && env NSUnbufferedIO=YES xcodebuild build-for-testing -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES | xcpretty
# Test
set -o pipefail && env NSUnbufferedIO=YES xcodebuild test-without-building -scheme $SCHEME -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" -enableCodeCoverage YES -resultBundlePath $RESULT_BUNDLE | xcpretty
Building and running tests with code coverage enabled.

The pre-clean section of the script makes sure that we do not have files present. The RESULT_BUNDLE is created by xcodebuild and RESULT_JSON later when processing the bundle. Although we could skip deleting the RESULT_JSON then on the other hand RESULT_BUNDLE must always be deleted or otherwise xcodebuild will log a warning and wouldn’t create a new one. The run tests section is separated into two steps: build the package for testing and then running tests without building. An alternative would be to use test argument instead of build-for-testing and test-without-building. The current setup makes it easier to see which step failed: was it a build error or a test error. Code coverage is enabled by passing enableCodeCoverage=YES and specifying a path where the result bundle should be created with resultBundlePath. Other things to note is that we are using set -o pipefail for making sure the script wouldn’t ignore any errors and would fail immediately. Also, we’ll pipe xcodebuild’s output to xcpretty for a bit nicer output in logs. If we run this piece of script then we end up with a CodeCoverage.xcbundle which contains code coverage information in the package root.

Extracting line coverage and forcing min coverage

For extracting code coverage information from the CodeCoverage.xcresult bundle we can use Apple’s xccov command line tool which was built exactly for that. The first step is that we’ll convert CodeCoverage.xcresult to CodeCoverage.json file with xccov.

CodeCoverage.json

The second step is to parse the create json file which lists all the targets and their code coverage. I am not fully sure why, but it contains two targets with the same name IndexedDataStore. One target contains correct code coverage, but the other one is just an empty target but with the same name. Therefore we’ll need to filter out the empty one which luckily is not so difficult with jq command line tool. The jq command first takes an array for key targets and then finds a target which has a name “IndexedDataStore” and value for key executableLines is greater than 0. After that, we can access the key lineCoverage in the found target. Finally, we’ll convert the code coverage to percentages, but as the value is in float then we’ll need to use the bc command.

set -o pipefail && env NSUnbufferedIO=YES xcrun xccov view –report –json $RESULT_BUNDLE > $RESULT_JSON
CODE_COVERAGE=$(cat $RESULT_JSON | jq '.targets[] | select( .name == "IndexedDataStore" and .executableLines > 0 ) | .lineCoverage')
CODE_COVERAGE=$(echo $CODE_COVERAGE*100.0 | bc)
Extracting line coverage from the xcresult.

After extracting the code coverage, we can proceed with forcing a minimum code coverage. Yet again, we’ll use bc command when comparing two float values and then checking if the comparison was true. If code coverage is too low, we’ll print red error message and in case it was enough, we’ll print success message in green colour.

COVERAGE_PASSES=$(echo "$CODE_COVERAGE > $MIN_CODE_COVERAGE" | bc)
if [ $COVERAGE_PASSES -ne 1 ]; then
printf "\033[0;31mCode coverage %.1f%% is less than required %.1f%%\033[0m\n" $CODE_COVERAGE $MIN_CODE_COVERAGE
exit -1
else
printf "\033[0;32mCode coverage is %.1f%%\033[0m\n" $CODE_COVERAGE
fi
Printing code coverage result.
Success message when min code coverage is 50%.
Error message when min code coverage requirement is 90%.

Calling the script from GitHub action

The created script can now be called from a GitHub action.

name: CI
on:
push:
branches: [ main ]
pull_request:
branches:
'*'
jobs:
tests:
runs-on: macos-latest
steps:
uses: actions/checkout@v2
name: Lint
run: swiftlint
name: Build for macOS
run: swift build -v
name: Run macOS tests
run: swift test -v
name: Run iOS tests
run: sh ./Scripts/ci-ios-code-coverage.sh
view raw ci.yml hosted with ❤ by GitHub
GitHub action which calls the created script.
GitHub action log which ran the script.

Summary

In this blog post we revisited code coverage set up for Swift packages. By using xcodebuild and xccov we created code coverage setup for Swift packages.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Categories
Swift Swift Package

Running tests in Swift package with GitHub actions

Some time ago I published a tiny Swift package IndexedDataStore which tackles a problem of storing data blobs on disk. It could the image data or anything else. When working with Swift packages then it is extremely easy to build and run tests on macOS but when we want to build and run tests on iOS simulator then we need to drop using swift build and swift test commands. Fortunately xcodebuild can help here and we can build and run tests without generating a project file ourselves with swift package generate-xcodeproj. Something I would like to mention about the generate-xcodeproj command is that it generates a project which does not contain resource files. The aim of this blog post is to configure a GitHub action which runs unit-tests both on macOS and iOS.

GitHub actions need to be in the repostory’s .github/workflows folder. GitHub workflows are defined in YAML files which need to be in that folder. Let’s jump into it and start creating a worflow which runs tests on macOS and iOS. GitHub workflows must define a name, in our case, name: CI. Secondly we’ll need to define when the workflow should be triggered. Running the workflow whenever pull request is created of when something was pushed to main branch will suffice.

name: CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches:
      - '*'

The next step is creating a job and defining a number of steps. Jobs require a name, then which resource it needs to use. The first step is the repository checkout followed by running swiftlint which is already part of the enviornment. The linting step uses the default configuration file .swiftlint.yml at the repository root. At this point we are ready to build and run tests.

jobs:
  unit_tests:
    runs-on: macos-latest
    steps:
    - name: Repository checkout
      uses: actions/checkout@v2
    - name: Lint
      run: swiftlint

The simplest way to build and test Swift packages is to use swift build and swift test commands which builds and runs tests on macOS. Unfortunately there is not a way to use those commands and setting the deplyoment to iOS. Therefore we’ll need to use xcodebuild command instead. It will know how to deal with Swift packages and therefore we only need to define scheme and destination arguments. It also makes sense to separate building (build-for-testing) from running tests (test-without-building) as it makes it easier to see where a failure happened. We can make the xcodebuild output a little bit cleaner by using xcpretty. In addition it is also reasonable to use set -o pipefail which makes the pipeline to use an occured error code as the final code of the whole pipeline. Finally, we’ll disable any buffering by setting NSUnbufferedIO to YES.

jobs:
  unit_tests:
    runs-on: macos-latest
    steps:
    - name: Repository checkout
      uses: actions/checkout@v2
    - name: Lint
      run: swiftlint
    - name: Build for macOS
      run: swift build -v
    - name: Run macOS tests
      run: swift test -v
    - name: Build for iOS
      run: set -o pipefail && env NSUnbufferedIO=YES xcodebuild build-for-testing -scheme IndexedDataStore -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" | xcpretty
    - name: Run iOS tests
      run: set -o pipefail && env NSUnbufferedIO=YES xcodebuild test-without-building -scheme IndexedDataStore -destination "platform=iOS Simulator,OS=latest,name=iPhone 12" | xcpretty

This completes our workflow and we can check in the YAML file which gets then picked up by GitHub. All the actions can be see under the Actions section on GitHub.

Summary

We looked into how to add a GitHub workflow for running tests both on macOS and iOS. Although swift build does not support building the package for iOS we could still do that with xcodebuild.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.