Sourcery is a code generation tool for Swift. While we’ve talked about code generation on the podcast, I haven’t really talked much about it on this blog. Today, I’d like to go in-depth on a concrete example of how we’re using Sourcery in practice for an app.
The app in question uses structs for its model layer. The app is mostly read-only, and data comes down from JSON, so structs work well. However, we do need to persist the objects so they load faster the next time, and so we need NSCoding conformance.
Swift 4 will bring Codable, a new protocol that supports JSON encoding and decoding, as well as NSCoding. Using Codable with NSKeyedArchiver is a little different than you’re used to, but it basically works. I’ve written up a small code sample here that you can drop into a playground to test. While that will obviate this specific use case for codegen eventually, the technique is still useful in the abstract. The new Codable protocol works by synthesizing encoding and decoding implementations for your types, and until we get access to this machinery directly, Sourcery is the best way to steal this power for ourselves. (Update, August 2018: We have moved away from this specific approach and towards Codable. It’s pretty good.)
To persist structs, I’m using the technique that I lay in this blog post. Essentially, each struct that needs to be encodable will get a corresponding NSObject wrapper that conforms it to NSCoding. If you haven’t read that post, now is a good time. The background in that post is necessary for the approach detailed here.
The technique in that blog post somewhat pedantically subscribes to the single responsiblity principle. One type (the struct) stores the data in memory, and another type (the NSObject wrapper) adds the conformance to NSCoding. The downside to this seperation is that you have to maintain a second type: if you add a new property to one of your structs, you need to add it to the init(coder:) and encode(with:) methods manually. The upside is that the separate type can be really easily generated.
This is where Sourcery comes in. With Sourcery, you use a templating language called Stencil to define templates. When the app is built, one of the build phases “renders” these templates into actual Swift code that is then compiled into your app. Other blog posts go into detail about how to set Sourcery up, so I won’t go into detail here, except to say that we check the Sourcery binary into git (so that everyone is on the same version) and the generated source files as well. Like CocoaPods, it’s just easier if those files are checked in.
Let’s discuss the actual technique. First, you need a protocol called AutoCodable.
protocol AutoCodable { }
This protocol doesn’t have anything in its definition — it purely serves as a signal to Sourcery that this file should have an encoder generated for it.
In a new file called AutoCodable.stencil, you can enumerate the objects that conform to this protocol.
{% for type in types.implementing.AutoCodable %}
// ...
{% endfor %}
Inside this for loop, we have access to a variable called type that has various properties that describe the type we’re working on this on this iteration of the for loop.
Inside the for loop, we can begin generating our code:
class Encodable{{ type.name }}: NSObject, NSCoding {
var {{ type.name | lowerFirstWord }}: {{ type.name }}?
init({{ type.name | lowerFirstWord }}: {{ type.name }}?) {
self.{{ type.name | lowerFirstWord }} = {{ type.name | lowerFirstWord }}
}
required init?(coder decoder: NSCoder) {
// ...
}
func encode(with encoder: NSCoder) {
// ...
}
}
Sourcery’s templates mix Swift code (regular code) with stencil code (meta code, code that writes code). Anything inside the double braces ({{ }}) will be printed out, so
class Encodable{{ type.name }}: NSObject, NSCoding
will output something like
class EncodableCoordinate: NSObject, NSCoding
Stencil and Sourcery also provide useful “filters”, like lowerFirstWord. That filter turns an upper-camel-case identifier into a lower-camel-case identifier. For example, it will convert DogHouse to dogHouse.
Thus, the line
var {{ type.name | lowerFirstWord }}: {{ type.name }}?
converts to
var coordinate: Coordinate?
which is exactly what we are after.
At this point, we can run our app, have the build phase generate our code, and take a look at the file AutoCodable.generated.swift file and ensure everything is generating correctly.
Next, let’s take a look at the init(coder:) function that we will have to generate. This is tougher. Let’s lay out the groundwork:
required init?(coder decoder: NSCoder) {
{% for variable in type.storedVariables %}
// ...
{% endfor %})
{{ type.name | lowerFirstWord }} = {{ type.name }}(
{% for variable in type.storedVariables %}
{{ variable.name }}: {{ variable.name }}{% if not forloop.last %},{% endif %}
{% endfor %}
)
}
We will loop through all of the variables in order to pull something useful out of the decoder. The last 4 lines here use an initializer to actually initialize the type from all the variables we will create. It will generate code something like:
coordinate = Coordinate(
latitude: latitude,
longitude: longitude
)
This corresponds to the memberwise initializer that Swift provides for structs. While working on this feature, I became worried that the user of this template could become confused if the memberwise initailizer disappeared or if they re-implemented it with the parameters in a different order. At the end of the post, we’ll take a look at a second Sourcery template for generating these initializers.
Because our model objects (“encodables”) can contain properties which are also encodables, we have to make sure to convert those to and from their encodable representations. For something like a coordinate, where the only values are two doubles (for latitude and longitude), we don’t need to do much. For more interesting objects, there are three cases of encodables (array, optional, and regular) to handle in a special way, so in total, we have 5 situations to handle: an encodable array, an encodable optional, a regular encodable, a regular optional, and a regular value.
required init?(coder decoder: NSCoder) {
{% for variable in type.storedVariables %}
{% if variable.typeName.name|hasPrefix:"[" %} // note, this doesn't support Array<T>, only `[T]`
// handle arrays
{% elif variable.isOptional and variable.type.implements.AutoCodable %}
// handle encodable optionals
{% elif variable.isOptional and not variable.type.implements.AutoCodable %}
// handle regular optionals
{% elif variable.type.implements.AutoCodable %}
// handle regular encodables
{% else %}
// handle regular values
{% endif %}
{% endfor %}
}
This sets up our branching logic.
Next, let’s look at each of the 5 cases. First, arrays of encodables:
guard let encodable_{{ variable.name }} = decoder.decodeObject(forKey: "{{ variable.name }}") as? [Encodable{{ variable.typeName.name|replace:"[",""|replace:"]","" }}] else { return nil }
let {{ variable.name }} = encodable_{{ variable.name }}.flatMap({ $0.{{ variable.typeName.name|replace:"[",""|replace:"]",""| lowerFirstWord}} })
The first line turns decodes an array of encodables, and the second line converts the encodables (which represent the NSCoding wrappers) into the actual objects. The code that’s generated looks something like:
guard let encodable_images = decoder.decodeObject(forKey: "image") as? [EncodableImage] else { return nil }
let images = encodable_images.flatMap({ $0.image })
Frankly, the stencil code is hideous. Stencil doesn’t support things like assignment of processed data to new variables, so things like {{ variable.typeName.name|replace:"[",""|replace:"]","" }} (which extracts the type name from the array’s type name) can’t be factored out. Stencil is designed more for presentation and less for logic, so this omission is understandable, however, it does make the code uglier.
The astute reader will note that I used underscores in a variable name, which is not the typical Swift style. I did this purely out of laziness: I didn’t want to deal with correctly capitalizing the variable name. Ideally, no one will ever look at this code, and it will work transparently in the background.
Next up, optionals. Encodable optionals first:
let encodable_{{ variable.name }} = decoder.decodeObject(forKey: "{{ variable.name }}") as? Encodable{{ variable.unwrappedTypeName }}
let {{ variable.name }} = encodable_{{variable.name}}?.{{variable.name}}
which generates something like
let encodable_image = decoder.decodeObject(forKey: "image") as? EncodableImage
let image = encodable_image?.image
And regular optionals, for things like numbers:
let {{ variable.name }} = decoder.decodeObject(forKey: "{{ variable.name }}") as? {{ variable.unwrappedTypeName }}
And its generated code:
let imageCount = decode.decodeObject(forKey: "imageCount") as? Int
Next, regular encodables:
guard let encodable_{{ variable.name }} = decoder.decodeObject(forKey: "{{ variable.name }}") as? Encodable{{ variable.typeName }},
let {{ variable.name }} = encodable_{{ variable.name }}.{{ variable.name }} else { return nil }
which generates:
guard let encodable_images = decoder.decodeObject(forKey: "image") as? EncodableImage,
let images = encodable_image?.image else { return nil }
And finally regular values:
guard let {{ variable.name }} = decoder.decodeObject(forKey: "{{ variable.name }}") as? {{ variable.typeName }} else { return nil }
And its generated code:
guard let imageCount = decoder.decodeObject(forKey: "imageCount") as? Int else { return nil }
I won’t go into too much detail on these last few, since they work similarly to the ones above.
Next, let’s quickly look at the encode(coder:) method. Because NSCoding is designed for Objective-C and everything is “optional” in Objective-C, we don’t have to handle optionals any differently. This means the number of cases we have to deal with are minimized.
func encode(with encoder: NSCoder) {
{% for variable in type.storedVariables %}
{% if variable.typeName.name|hasPrefix:"[" %}
// array
{% elif variable.type.implements.AutoCodable %}
// encodable
{% else %}
// normal value
{% endif %}
{% endfor %}
}
The array handling code looks like this:
let encoded_{{ variable.name }} = {{ type.name | lowerFirstWord }}?.{{ variable.name }}.map({ return Encodable{{ variable.typeName.name|replace:"[",""|replace:"]","" }}({{ variable.typeName.name|replace:"[",""|replace:"]",""| lowerFirstWord }}: $0) })
encoder.encode(encoded_{{ variable.name }}, forKey: "{{ variable.name }}")
The encodable handling code looks like this:
encoder.encode(Encodable{{ variable.unwrappedTypeName }}({{ variable.name | lowerFirstWord }}: {{ type.name | lowerFirstWord }}?.{{ variable.name }}), forKey: "{{ variable.name }}")
And the normal properties can be encoded like so:
encoder.encode({{ type.name | lowerFirstWord }}?.{{ variable.name }}, forKey: "{{ variable.name }}")
The stencil code is a bit complex, hard to read, and messy. However, the nice thing is that if you write this repetitive code once, it will generate a lot more repetitive code for you. For example, the whole template is 86 lines, and for all of our models, it generates about 500 lines of boilerplate.
One thing I was surprised to learn is that the code is surprisingly robust. We wrote this template in a day back in February. In the intervening 6 months, we haven’t edited this template at all. No modifications have been necessary to support any of the changes we made to our models since then.
The last thing we need to support a few protocol that show our system how to bridge between the encodables and the structs:
extension Encodable{{ type.name }}: Encoder {
typealias Value = {{ type.name }}
var value: {{ type.name }}? {
return {{ type.name | lowerFirstWord }}
}
}
extension {{ type.name }}: Archivable {
typealias Encodable = Encodable{{ type.name }}
var encoder: Encodable{{ type.name }} {
return Encodable{{ type.name }}({{ type.name | lowerFirstWord }}: self)
}
}
To learn more about the purpose of these extra conformances, you can read the original blog post.
Finally, let’s take a quick look at the Sourcery template for something we could call AutoInitializable. The approach is similiar to what we’ve looked at for AutoCodable, with one additional component. Because of Swift’s initialization rules, it only works with structs (classes in Swift can’t add new initializers in extensions).
{# this template currently only works with structs (no classes) #}
{% for type in types.implementing.AutoInitializable %}
extension {{ type.name }} {
{% if type.kind == "struct" and type.initializers.count == 0 %}
// no initializer, since there is a free memberwise intializer that Swift gives us
{% else %}
{{ type.accessLevel }} init({% for variable in type.storedVariables %}{{ variable.name }}: {{ variable.typeName }}{% if variable.annotations.initializerDefault %} = {{ variable.annotations.initializerDefault }}{% endif %}{% if not forloop.last %}, {% endif %}{% endfor %}) {
{% for variable in type.storedVariables %}
self.{{ variable.name }} = {{ variable.name }}
{% endfor %}
}
{% endif %}
}
{% endfor %}
I won’t belabor this with a line-by-line breakdown, but I will note that it takes advantage of a Sourcery feature called “annotations”. This lets you add additional information to a particular property. In this case, we had certain cases where some properties needed to have initializerDefault values (usually nil), so we were able to add support for that. A Sourcery annotation is declared like so:
// sourcery: initializerDefault = nil
var distance: Distance?
This post lays out one of the more involved uses for code generation. Sourcery gives you the building blocks you need to build complex templates like this one, and it removes the need to maintain onerous boilerplate manually. This particular template code was designed for our needs and may not suit every app. It will ultimately be rendered obsolete by Swift 4’s Codable. However, the example serves as case study for more complex Sourcery templates. They are flexible without any loss in robustness. I was initially worried that this meta-code would be brittle, breaking frequently, but in practice, these templates haven’t required a single change since they were first written, and they ensure that our encodable representations always stay perfectly up-to-date with any model changes.
