Introducing YapDatabaseExtensions

Functional extensions for fun and profit

I am a big fan of YapDatabase. In another post, I'll go into more detail into why I think it's the best datasource technology for iOS. But for now, I’m going to assume you have some familiarity with it, as I want to talk about some pain points and my solutions to them, which I've open sourced as a Swift framework YapDatabaseExtensions.

Immutability & Swift

Swift is quickly approaching the 1st anniversary of its public release. I’m almost certain that we’ll see Swift 2.0 in June at WWDC ’15. So, writing about the difference between Objective-C and Swift might seem moot at this point. However, the majority of Cocoa developers I’ve spoken to are still not using Swift “full time”. Only a handful are shipping Swift code, and I suspect an even smaller handful, a small child’s hand maybe, are shipping apps which started in Swift. There is a big difference between adopting Swift in an extant Objective-C codebase and starting a new app with Swift.

When I first started writing Swift, I was basically writing Objective-C but with Swift syntax. It took me a little time to prefer structs over classes, and map over for loops. One of the biggest road blocks for me was that I was using YapDatabase for my persistence layer, and so my model types were all classes implementing NSCoding. Ironically, YapDatabase’s documentation stresses the best practice of treating models as immutable, yet I wasn’t able to architect for immutability with structs and constants because they wouldn’t then conform to NSCoding. Of course it only took a bit of Twitter talk to push me onto the right track:

This led me to start playing with archiving protocols.

/**
A generic protocol which acts as an archiver for value types.
*/
public protocol Archiver: NSCoding {
  typealias ValueType

  /// The value type which is being encoded/decoded
  var value: ValueType { get }

  /// Required initializer receiving the wrapped value type.
  init(_: ValueType)
}

Inspired by Andy's tweet, this is a protocol which an adaptor class would conform to. It is responsible for implementing NSCoding on behalf of its composed value type. Therefore each value type needs its own Archiver class. This requirement can also be codified in a protocol.

/**
A generic protocol which can be implemented to vend another
object capable of archiving the receiver.
*/
public protocol Saveable {
  typealias ArchiverType: Archiver

  /// The archive(r)
  var archive: ArchiverType { get }
}

Unfortunately, there is no way in Swift to constrain that the ValueType of the ArchiverType is Self in the Saveable protocol directly. Such a constraint can be defined using Swift's generic parameter clause however, as we'll see later.

Implementing these protocols on a value type will therefore allow us to serialise it, using the archive property. For example, assume a Product struct type.

extension Product: Saveable {

  public typealias Archive = ProductArchiver

  public var archive: Archive {
    return Archive(self)
  }
}

public class ProductArchiver: NSObject, NSCoding, Archiver {
  public let value: Product

  public required init(_ v: Product) {
    value = v
  }

  public required init(coder aDecoder: NSCoder) {
    // etc
    value = Product()
  }

  public func encodeWithCoder(aCoder: NSCoder) {
    // etc
  }
}

Putting all this together means that we can start modelling our application domain using immutable value types. To store them in YapDatabase we just need a way to write such types to the database. Enter YapDatabaseExtensions.

YapDatabaseExtensions

YapDatabase is a key-value store, it essentially serializes objects into data blobs and then stores them against an index. The index is composed of a collection and a key, both of which are strings. Internally YapDatabase does have a class to efficiently model this index, but we can do it more succinctly in Swift:

public struct YapDB { }
extension YapDB {

    public struct Index {
        public let collection: String
        public let key: String

        public init(collection: String, key: String) {
            self.collection = collection
            self.key = key
        }
    }
}

And saving AnyObject against an index is straightforward. We can write a function which accepts a YapDB.Index argument with the associated object and metadata, using YapDatabase's setObject(:forKey:inCollection:) API.

extension YapDatabaseReadWriteTransaction {
    func writeAtIndex(index: YapDB.Index, object: AnyObject, metadata: AnyObject? = .None) {
        if let metadata: AnyObject = metadata {
            setObject(object, forKey: index.key, inCollection: index.collection, withMetadata: metadata)
        }
        else {
            setObject(object, forKey: index.key, inCollection: index.collection)
        }
    }
}

But in code, we want reading and writing our value types to be declarative and obvious, like this:

let product = Product()
transaction.write(product)

We therefore need another protocol, to create database indexes from types. The Persistable protocol, listed below, will force instances of types to be stored in the same collection. While this isn't strictly enforced by YapDatabase it is nevertheless a best practice as it can help to keep YapDatabaseViews efficient. Additionally, it means that types must have a way of defining their own uniqueness, their own identity. So, it's nicer to separate this out into two protocols.

/**
A generic protocol which is used to return a unique identifier
for the type.
*/
public protocol Identifiable {
    typealias IdentifierType: Printable
    var identifier: IdentifierType { get }
}

public protocol Persistable: Identifiable {
    /// The YapDatabase collection name the type is stored in.
    static var collection: String { get }
}

With these protocols in place, we are now in a position to define a generic write function on YapDatabaseReadWriteTransaction.

extension YapDatabaseReadWriteTransaction {
  /**
  Writes a Persistable value, conforming to Saveable to the database inside the read write transaction.

  :param: value A Value.
  :returns: The Value.
  */
  public func write<Value where Value: Saveable, Value: Persistable, Value.ArchiverType.ValueType == Value>(value: Value) -> Value {
      writeAtIndex(indexForPersistable(value), object: value.archive)
      return value
  }
}

It's worth stepping through the generic parameter clause to this function.

  • We define the generic parameter, Value, which is the type of its only argument (value), and also the return type. The where clause defines the constraints on this type.
  • It must be Saveable so that the archive can be accessed/created.
  • It must be Persistable so that an index can be generated.
  • Lastly, there is the constraint that the Value's ArchiverType's ValueType is equal to Value itself. Essentially this validates that the archiver returned from archive is of the correct type.

Reading value from YapDatabase is very similar. We can read an object given an index.

extension YapDatabaseReadTransaction {
  /**
  Reads the object sored at this index using the transaction.
      
  :param: index The YapDB.Index value.
  :returns: An optional AnyObject.
  */
  public func readAtIndex(index: YapDB.Index) -> AnyObject? {
      return objectForKey(index.key, inCollection: index.collection)
  }
}

But, in order to maintain type safety, we need to unarchive the AnyObject, assuming it's an ArchiverType we can attempt to get the value.

public func valueFromArchive<Value where Value: Saveable, Value.ArchiverType.ValueType == Value>(archive: AnyObject?) -> Value? {
    return archive.map { ($0 as! Value.ArchiverType).value }
}

Here, we use map to transform AnyObject? into Value?. If the argument is .None the optional will return .None, else it will cast the object into the type of archiver Value defines. And if that is successful, it will return the decoded value. Note that this will fail at runtime if attempted to extract a value from the wrong archive type. This function makes implementing NSCoding with nested value types much easier.

When it comes to reading directly out of the database, we now have all the pieces to do this easily and elegantly.

/**
Unarchives a value type if stored at this index

:param: index The YapDB.Index value.
:returns: An optional Value.
*/
public func readAtIndex<Value where Value: Saveable, Value: Persistable, Value.ArchiverType.ValueType == Value>(index: YapDB.Index) -> Value? {
    return valueFromArchive(readAtIndex(index))
}

This covers the basis of the functions provided in YapDatabaseExtensions. There is additional support for reading and writing objects (i.e. NSCoding classes), and a combination of object or value type metadata.

Functional APIs & YapDatabaseConnection

YapDatabase is a transaction based persistence framework. Transactions provide the read and write API which we've extended in the discussion above. And they are executed in blocks on queues in database connections. This means that to read from the database the programmer must create a connection, define a variable and initialize it from within a transaction block on the connection. This architecture doesn't suit itself to functional paradigms as the read transaction doesn't return anything. Therefore, we provide our own function for reading on YapDatabaseConnection.

extension YapDatabaseConnection {
  /**
  Synchronously reads from the database on the connection. The closure receives
  the read transaction, and the function returns the result of the closure. This
  makes it very suitable as a building block for more functional methods.

  :param: block A closure which receives YapDatabaseReadTransaction and returns T
  :returns: An instance of T
  */
  public func read<T>(block: (YapDatabaseReadTransaction) -> T) -> T {
      var result: T! = .None
      readWithBlock { result = block($0) }
      return result
  }

Now it is trivial to implement the same read & write API which we've defined for transactions on connections and the database directly. Although reading values directly from the database is discouraged and only provided for ease of use and testing.

extension YapDatabaseConnection {
  /**
  Synchronously reads the Value sored at this index using the connection.
  :param: index The YapDB.Index value.
  :returns: An optional Value.
  */
  public func readAtIndex<Value where Value: Saveable, Value: Persistable, Value.ArchiverType.ValueType == Value>(index: YapDB.Index) -> Value? {
      return read { $0.readAtIndex(index) }
  }
}

In conclusion, some simple protocols and generic constraints have allowed us to adopt Swift's immutable and value-type paradigms, but still take full advantage of rock solid Objective-C persistence frameworks.

Posted on Apr 26
Written by Daniel Thorpe