Categories
iOS Swift SwiftUI

AsyncPhoto with caching in SwiftUI (part 2)

In the part 1 of the series, AsyncPhoto for displaying large photos in SwiftUI, we built a SwiftUI view which has a similar interface to Apple’s AsyncImage, but provides a way to use any kind of image data source. In the part 2 of the series, we’ll implement an in-memory cache for the AsyncPhoto. This is important for reducing any flickering caused by the nature of async image loading. An example to highlight where it comes useful is when we have a detail view which displays a thumbnail of a large photo. If we open the detail view multiple times for the same photo, we really do not want to see the loading spinner every single time. Another benefit is that we do not need to load a huge photo in memory and then spending CPU on scaling it down.

OK, let’s jump into it.

The aim of the cache is to cache the scaled down images. We never want to cache the original image data since it would make the memory usage to through the roof, and we would still need to use CPU to scale down the image. Before we start, we need to remember that in part 1 we designed the AsyncPhoto in a way where it has an ID, scaledSize properties and a closure for returning image data asynchronously. Therefore, the caching key needs to be created by using the ID and the scaled size, since we might want to display a photo in multiple AsyncPhoto instances with different sizes. Let’s create an interface for the caching layer. We’ll go for a protocol based approach, which allows replacing the caching logic with different concrete implementations. In this blog post we’ll go for a NSCache backed caching implementation, but anyone else could use other approaches as well, like LRUCache.

/// An interface for caching images by identifier and size.
protocol AsyncPhotoCaching {
/// Store the specified image by size and identifier.
/// – Parameters:
/// – image: The image to be cached.
/// – id: The unique identifier of the image.
func store(_ image: UIImage, forID id: any Hashable)
/// Returns the image associated with a given id and size.
/// – Parameters:
/// – id: The unique identifier of the image.
/// – size: The size of the image stored in the cache.
/// – Returns: The image associated with id and size, or nil if no image is associated with id and size.
func image(for id: any Hashable, size: CGSize) -> UIImage?
/// Returns the caching key by combining a given image id and a size.
/// – Parameters:
/// – id: The unique identifier of the image.
/// – size: The size of the image stored in the cache.
/// – Returns: The caching key by combining a given id and size.
func cacheKey(for id: any Hashable, size: CGSize) -> String
}
extension AsyncPhotoCaching {
func cacheKey(for id: any Hashable, size: CGSize) -> String {
"\(id.hashValue):w\(Int(size.width))h\(Int(size.height))"
}
}

The protocol only defines 3 functions for writing, reading, and creating a caching key. We’ll provide a default implementation for the cacheKey(for:size:) function. Since the same image data should be cached by size, the cache key combines id and size arguments. Since we are dealing with floats in a string, we’ll round the width and height.

The next step is to create a concrete implementation. In this blog post, we’ll go for NSCache which automatically evicts images from the cache in case of a memory pressure. The downside of a NSCache is that the logic in which order images are evicted is not defined. The implementation is straight-forward.

struct AsyncPhotoCache: AsyncPhotoCaching {
private var storage: NSCache<NSString, UIImage>
static let shared = AsyncPhotoCache(countLimit: 10)
init(countLimit: Int) {
self.storage = NSCache()
self.storage.countLimit = countLimit
}
func store(_ image: UIImage, forID id: any Hashable) {
let key = cacheKey(for: id, size: image.size)
storage.setObject(image, forKey: key as NSString)
}
func image(for id: any Hashable, size: CGSize) -> UIImage? {
let key = cacheKey(for: id, size: size)
return storage.object(forKey: key as NSString)
}
}

We also added a shared instance since we want to use a single cache instance for all the AsyncPhoto instances. Let’s see how the AsyncPhoto implementation changes when we add a caching layer. The answer is, not so much.

struct AsyncPhoto<ID, Content, Progress, Placeholder>: View where ID: Hashable, Content: View, Progress: View, Placeholder: View {
// redacted
init(id value: ID = "",
scaledSize: CGSize,
cache: AsyncPhotoCaching = AsyncPhotoCache.shared,
data: @escaping (ID) async -> Data?,
content: @escaping (Image) -> Content = { $0 },
progress: @escaping () -> Progress = { ProgressView() },
placeholder: @escaping () -> Placeholder = { Color(white: 0.839) }) {
// redacted
}
var body: some View {
// redacted
}
@MainActor func load() async {
// Here we access the cache
if let image = cache.image(for: id, size: scaledSize) {
phase = .success(Image(uiImage: image))
}
else {
phase = .loading
if let image = await prepareScaledImage(for: id) {
guard !Task.isCancelled else { return }
phase = .success(image)
}
else {
guard !Task.isCancelled else { return }
phase = .placeholder
}
}
}
private func prepareScaledImage(for id: ID) async -> Image? {
guard let photoData = await data(id) else { return nil }
guard let originalImage = UIImage(data: photoData) else { return nil }
let scaledImage = await originalImage.scaled(toFill: scaledSize)
guard let finalImage = await scaledImage.byPreparingForDisplay() else { return nil }
// Here we store the scaled down image in the cache
cache.store(finalImage, forID: id)
return Image(uiImage: finalImage)
}
}

We added a new cache argument but also set the default value to the shared instance. The load() function tries to read a cached image as a first step, and the preparedScaledImage(for:) updates the cache. We rely on the cache implementation to keep the cache size small, therefore here is no code for manually evicting images from the cache when the ID changes. The main reason is that the AsyncPhoto instance does not have enough context for deciding this. For example, there might be other instances showing the photo for the old ID or maybe a moment later we want to display the photo for the old ID.

To recap, what we did. We defined an interface for caching images, created a NSCache based in-memory cache and hooked it up to the AsyncPhoto. We did all of this in a way that we did not need to change any existing code using AsyncPhoto instances.

There were some other tiny improvements, like using Task.isCancelled() to more quickly react to the ID change, setting the default placeholder colour to a light gray, and providing a default implementation for the content closure. Please check the example project for the full implementation. Here is the example project which reloads an avatar and as we can see at first, spinner is shown, but when images are cached, the change is immediate.

SwiftUIAsyncPhotoExample2 (GitHub, Xcode 15.1)

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Categories
iOS macOS Swift

Building a memory cache for a file reader in Swift

In the previous blog post Reading data from a file with DispatchIO I built a small FileReader which enabled reading data from a file for random byte ranges. Signal Path uses a similar file reader but in addition it also caches read data in some interactions where there are a lot of read requests active, for example, while recording. Therefore, in this blog post we’ll build a small file data cache which stores data chunks for byte ranges. The cache needs to be performant, otherwise it would be faster to just read the data from the file if it is slow.

Defining an interface for the cache

The interface of the FileMemoryCache needs to provide a way for retrieving data, storing data, and limiting the overall memory usage.

final class FileMemoryCache {
init(byteCountLimit: Int)
/// Returns data for the byte range
func data(for byteRange: CountableRange<Int>) -> DispatchData?
/// Caches data for the byte range and evicts older data when exceeding byte count limit
func set(_ data: DispatchData, byteRange: CountableRange<Int>)
/// Removes everything from the cache
func removeAll()
}

Storing and retrieving cached data

The strategy of caching is to use a sorted array where the array is sorted by the first index of a byte range. Items in the array are structs containing a byte range and data. Having a sorted array makes it easier to merge data chunks when retrieving data for a longer byte range than the individual array items provide. As the aim is to keep it as fast as possible then data is merged only when cache is accessed which reduces the need to copy memory multiple times. The downside is that the cache can store overlapping chunks of data which reduces the efficiency. But on the other hand it is fast. Sorted array also enables to use binary search if linear search happens to be too slow with large amount of cached data chunks.

final class FileMemoryCache {
private var sortedItems = [StorageItem]()
struct StorageItem {
let byteRange: CountableRange<Int>
let data: DispatchData
}
}

Let’s start with the store method which takes a data and a byte range it corresponds to. For keeping the array sorted, we’ll use the first index of the byte range as the attribute for sorting. We’ll also make sure that the current item at the insertion index does not already contain the byte range we are trying to cache. If it does contain, we can safely ignore the current store request as the data is already cached. Additionally, we’ll check if the previous item in the array as it might also contain the byte range.

func set(_ data: DispatchData, byteRange: CountableRange<Int>) {
if let insertionIndex = sortedItems.firstIndex(where: { $0.byteRange.startIndex >= byteRange.startIndex }) {
let canDiscard: Bool = {
guard !sortedItems[insertionIndex].byteRange.contains(byteRange) else { return true }
guard insertionIndex > sortedItems.startIndex else { return false }
return sortedItems[insertionIndex.advanced(by: -1)].byteRange.contains(byteRange)
}()
if !canDiscard {
sortedItems.insert(StorageItem(byteRange: byteRange, data: data), at: insertionIndex)
}
}
else {
sortedItems.append(StorageItem(byteRange: byteRange, data: data))
}
}

When it comes to retrieving data then we’ll need to find intersecting cached data chunks and merge them if needed. Therefore, the first step is to collect cached data chunks which intersect with the requested range. For keeping it as simple as possible, will just loop over the array without using binary search and collect the items which are intersecting. The next step after that is to make sure the whole requested byte range is already present in the cache. This can be checked quite easily with IndexSet which contains indexes for every requested bytes. By removing byte indexes in intersecting ranges, the index set becomes empty. It means that every byte index is cached.

func data(for byteRange: CountableRange<Int>) -> DispatchData? {
// Find items which intersect with the search range
guard let firstIntersectingItemIndex = sortedItems.firstIndex(where: { byteRange.intersects($0.byteRange) }) else { return nil }
let intersectingItems = sortedItems[firstIntersectingItemIndex…].prefix(while: { byteRange.intersects($0.byteRange) })
// Check if the whole range is covered
var byteIndexesInCache = IndexSet(integersIn: byteRange)
intersectingItems.forEach({ byteIndexesInCache.remove(integersIn: $0.byteRange) })
guard byteIndexesInCache.isEmpty else { return nil }
// …
}

We’ll need to copy intersecting byte indexes into a final contiguous data. It could be that the whole cached data chunk can be copied over or just a part of it. After figuring out which part of the cached data chunk to copy, the next step is to calculate the byte index in the resulting memory region which starts with the zero byte index. The implementation can be seen below which includes byte range index conversions.

func data(for byteRange: CountableRange<Int>) -> DispatchData? {
// …
guard let byteRangePointer = malloc(byteRange.count) else { return nil }
for storageItem in intersectingItems {
let rangeToCopyInData: CountableRange<Int> = {
switch (byteRange.contains(storageItem.byteRange.startIndex), byteRange.contains(storageItem.byteRange.endIndex – 1)) {
case (true, true):
// Everything: |+++++++++++++|
return 0..<storageItem.byteRange.count
case (false, true):
// End: |—++++++++++|
return byteRange.startIndex – storageItem.byteRange.startIndex..<storageItem.byteRange.count
case (true, false):
// Start: |++++++++++—|
return 0..<byteRange.endIndex – storageItem.byteRange.startIndex
case (false, false):
// Middle: |—+++++++—|
return byteRange.startIndex – storageItem.byteRange.startIndex..<byteRange.endIndex – storageItem.byteRange.startIndex
}
}()
// Find the byte range in the allocated memory range where to copy the cached data section
let destinationByteIndex = storageItem.byteRange.startIndex – byteRange.startIndex + rangeToCopyInData.startIndex
let destinationPointer = byteRangePointer.advanced(by: destinationByteIndex)
let destinationBufferPointer = UnsafeMutableRawBufferPointer(start: destinationPointer, count: rangeToCopyInData.count)
storageItem.data.copyBytes(to: destinationBufferPointer, from: rangeToCopyInData)
// Ignore remaining intersections because everything is copied
if destinationByteIndex + rangeToCopyInData.count >= byteRange.endIndex {
break
}
}
let byteRangeBufferPointer = UnsafeRawBufferPointer(start: byteRangePointer, count: byteRange.count)
return DispatchData(bytesNoCopy: byteRangeBufferPointer, deallocator: .free)
}

Summary

We built a simple cache for storing data chunks for byte ranges. We used a sorted array for storing individual data chunks which made it easier to merge them into one when the cache is accessed. Make sure to check the full implementation on GitHub which also includes evicting cached data.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Example Project

FileReaderPlayground (Xcode 12.4)

Categories
iOS LinkPresentation macOS Swift UIKit

Loading URL previews using LinkPresentation framework in Swift

iOS and macOS got a new framework in WWDC’19 named LinkPresentation. LinkPresentation enables fetching URL previews.

Adding LPLinkView for presenting preview

LPLinkView is a view subclass meant for rendering LPLinkMetadata. LPLinkMetadata contains information about the link: title, icon, image, video.

import LinkPresentation
let linkView = LPLinkView(metadata: LPLinkMetadata())
linkView.translatesAutoresizingMaskIntoConstraints = false
stackView.insertArrangedSubview(linkView, at: 0)
Adding LPLinkView to stack view

Fetching previews with LPMetadataProvider

Instances of LPLinkMetadata are fetched using LPLinkMetadataProvider for a given url. LPLinkMetadata is conforming to NSSecureCoding what enables a way of converting it to Data and storing the metadata on disk. Next time we need metadata for this url, we can use a locally cached data instead. Archiving and unarchiving is done with help of NSKeyedArchiver and NSKeyedUnarchiver and in the example archived data is stored in UserDefaults. In real apps it makes sense to store the data in separate files instead and not polluting UserDefaults with preview data.

private let metadataStorage = MetadataStorage()
private lazy var metadataProvider = LPMetadataProvider()
private weak var linkView: LPLinkView?
@IBAction func loadPreview(_ sender: UIButton) {
if let text = textField.text, let url = URL(string: text) {
// Avoid fetching LPLinkMetadata every time and archieve it disk
if let metadata = metadataStorage.metadata(for: url) {
linkView?.metadata = metadata
return
}
metadataProvider.startFetchingMetadata(for: url) { [weak self] (metadata, error) in
if let error = error {
print(error)
}
else if let metadata = metadata {
DispatchQueue.main.async { [weak self] in
self?.metadataStorage.store(metadata)
self?.linkView?.metadata = metadata
}
}
}
}
}
Fetching and caching LPMetadata
struct MetadataStorage {
private let storage = UserDefaults.standard
func store(_ metadata: LPLinkMetadata) {
do {
let data = try NSKeyedArchiver.archivedData(withRootObject: metadata, requiringSecureCoding: true)
var metadatas = storage.dictionary(forKey: "Metadata") as? [String: Data] ?? [String: Data]()
while metadatas.count > 10 {
metadatas.removeValue(forKey: metadatas.randomElement()!.key)
}
metadatas[metadata.originalURL!.absoluteString] = data
storage.set(metadatas, forKey: "Metadata")
}
catch {
print("Failed storing metadata with error \(error as NSError)")
}
}
func metadata(for url: URL) -> LPLinkMetadata? {
guard let metadatas = storage.dictionary(forKey: "Metadata") as? [String: Data] else { return nil }
guard let data = metadatas[url.absoluteString] else { return nil }
do {
return try NSKeyedUnarchiver.unarchivedObject(ofClass: LPLinkMetadata.self, from: data)
}
catch {
print("Failed to unarchive metadata with error \(error as NSError)")
return nil
}
}
}
Local LPMetadata storage using UserDefaults as storage

Summary

LinkPresentation framework adds a easy way of fetching previews for web pages. It provides a LPLinkView class making it extremely easy to render the preview.

If this was helpful, please let me know on Mastodon@toomasvahter or Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Example Project

LinkPresentationView (Xcode 11 GM)