Reading data from a file with DispatchIO

Signal Path is an app which works with large files, even in several gigabytes. The app reads ranges from the file and visualizes the data. Signal Path heavily relies on DispatchIO for having efficient access to data in a file. The aim of the blog post is to build a FileReader class which wraps DispatchIO and provides a similar functionality.

DispatchIO manages a file descriptor and coordinates file accesses. DispatchIO object can be created by providing a path to the file and specifying the stream type: stream or random access. In the context of this post we are interested in random access as we would like to read random ranges of bytes from a file. Therefore, let’s start with defining an interface for the FileReader.

final class FileReader {
init(fileURL: URL)
/// Opens the I/O channel with random access semantics
func open()
/// Closes the opened I/O channel
func close()
/// Reads data at byte range.
func read(byteRange: CountableRange<Int>, queue: DispatchQueue = .main, completionHandler: @escaping (DispatchData?) -> Void)
}
The interface for DispatchIO wrapping class.

The interface is pretty straight-forward with functions to open and close the file and with an asynchronous read function. DispatchIO returns read data as DispatchData which is a contiguous block of memory and what also can be cast into Data if needed.

We can use an init method on DispatchIO which has path argument when dealing with file paths. Note that the queue parameter on DispatchIO just specifies which queue is used for the cleanup handler, most of the cases .main will suffice. After creating the channel we can additionally control if data is returned once or partially with multiple handler callbacks. In our case, we would like to get a single callback, and therefore we need to set the low limit to Int.max. Then the read data is returned in with a single callback.

func open() -> Bool {
guard channel == nil else { return true }
guard let path = (fileURL.path as NSString).utf8String else { return false }
channel = DispatchIO(type: .random, path: path, oflag: 0, mode: 0, queue: .main, cleanupHandler: { error in
print("Closed a channel with status: \(error)")
})
// Load the whole requested byte range at once
channel?.setLimit(lowWater: .max)
guard self.channel != nil else { return false }
print("Opened a channel at \(fileURL)")
return true
}
func close() {
channel?.close()
channel = nil
}

Reading from a file requires having an opened channel and defining a byte range.

func read(byteRange: CountableRange<Int>, queue: DispatchQueue = .main, completionHandler: @escaping (DispatchData?) -> Void) {
if let channel = channel {
channel.read(offset: off_t(byteRange.startIndex), length: byteRange.count, queue: queue, ioHandler: { done, data, error in
print(done, data?.count ?? -1, error)
completionHandler(data)
})
}
else {
print("Channel is closed")
completionHandler(nil)
}
}

And the file reader can be used like this:

let fileURL = Bundle.main.url(forResource: "DataFile", withExtension: nil)!
let reader = FileReader(fileURL: fileURL)
if reader.open() {
reader.read(byteRange: 0..<20) { data in
if let data = data {
print("Read bytes: \(data.map({ UInt8($0) }))")
}
else {
print("Failed to read data")
}
}
}
else {
print("Failed to open")
}

Summary

DispatchIO provides an efficient way for accessing raw bytes in a file. Wrapping it into a FileReader class gives us a compact interface for working with file data. Please checkout the playground which contains the full implementation and the example: FileReaderPlayground.

If this was helpful, please let me know on Twitter @toomasvahter. Feel free to subscribe to RSS feed. Thank you for reading.

Example Project

FileReaderPlayground (Xcode 12.4)

1 Comment »

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s