October 5, 2015

Coordinators Redux

I wrote about coordinators at the beginning of the year, but the idea has matured a lot since then, and I'd like to reintroduce the topic with all of my learnings from the last few months.

This is adapted from a talk I gave at NSSpain this year. You can find the slides here. I'll post a link to the video when it's available as well.

Three Problems

Overstuffed App Delegates

Apple does a really poor job of guiding us into putting code in good places. It's really up to us to figure out how to structure our apps. The first place this is apparent is the app's delegate.

The app delegate is the entry point into any app. Its primary responsibility is shuttling messages back and forth from the operating system to the app's subsystems. Unfortunately, because of its position at the center of everything, it's extremely easy to just plop stuff in here. One casualty of this strategy is the root view controller's configuration. If you have a tab bar controller as the root of your app, you have to set up all the tab bar controller's children somewhere, and the app delegate is as good a place as any.

In the first app that I made (and I suspect this is true for many of my readers as well), I put all the set up for my root view controller right in my app delegate. That code doesn't really belong here, and it's only there out of convenience.

I realized that after I made my first app, and I grew wiser. I did this trick:

@interface SKTabBarController : UITabBarController

I would created a subclass of the root view controller I wanted to use, and I would tuck my code away in there. It was a temporary patch over the problem, but ultimately, it's not the right place for this code either. I would propose that we examine this object, the root view controller, from a perspective of responsiblities. Managing the child view controllers is within those duties, but allocating and configuration them not as much. We're subclassing a thing that was never intended to subclassed, just so we can hide away some code that doesn't have a home.

Coordinators are a better home for this app configuration logic.

Too Many Responsibilities

Here's another confounding problem. In the same way that its easy to dump tons of responsibilities into the app delegate, each individual view controller also suffers.

A small selection of the stuff we have view controllers do:

  1. Model-View Binding
  2. Subview Allocation
  3. Data Fetching
  4. Layout
  5. Data Transformation
  6. Navigation Flow
  7. User Input
  8. Model Mutation
  9. and many more besides

I came up with ways to tuck away these responsiblities in the children of each view controller in 8 Patterns to Help You Destroy Massive View Controller. All of these responsiblities can't be in one place. It's how we end up with 3000 line view controllers.

Which of this stuff should actually be in this class? Which should be elsewhere? What is the view controller's job? It's not clear.

There's a great quote by Graham Lee that I love.

When you get overly attached to MVC, then you look at every class you create and ask the question “is this a model, a view, or a controller?”. Because this question makes no sense, the answer doesn’t either: anything that isn’t evidently data or evidently graphics gets put into the amorphous “controller” collection, which eventually sucks your entire codebase into its innards like a black hole collapsing under its own weight.

What the hell is a view controller? Controllers in the Smalltalkian sense were originally intended strictly for user input. And even the word "control" screws us. As I've written before:

When you call something a Controller, it absolves you of the need to separate your concerns. Nothing is out of scope, since its purpose is to control things. Your code quickly devolves into a procedure, reaching deep into other objects to query their state and manipulate them from afar. Boundless, it begins absorbing responsibilities.

Which of those responsibilities should be at the "controller level"? Graham's right, the question doesn't even make sense. Because we're stuck with this awful word, we have to be really careful with what we allow our view controllers to do. If we don't, it's black hole central.

Coordinators can help us with this.

Smooth Flow

The last problem I want to discuss is navigation flow.

- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {
    id object = [self.dataSource objectAtIndexPath:indexPath];
    SKDetailViewController *detailViewController = [[SKDetailViewController alloc] initWithDetailObject:object];
    [self.navigationController pushViewController:detailViewController animated:YES];

This is a pretty common snippet of code, and I believe it's in the Apple templates as well. Unfortunately, it's garbage. Let's go through it line-by-line:

id object = [self.dataSource objectAtIndexPath:indexPath];

This first line is fine. The dataSource is a logical child of the view controller, and we're asking it for the object we need to refer to.

SKDetailViewController *detailViewController = [[SKDetailViewController alloc] initWithDetailObject:object];

This is where things start getting a little hairy. The view controller is instantiating a new view controller, the next one in the chain, and configuring it. The view controller "knows" what's coming up next in the flow. It knows how that thing is to be configured. The view controller that's doing the presenting knows a ton of detail about where it exists in the world of your app.

[self.navigationController pushViewController:detailViewController animated:YES];

The third line is where it totally goes off the rails. The view controller is now grabbing its parent, because remember, these view controllers exist in a heirarchy, and then it's sending a precise message to its parent about what to do. It's bossing its parent around. In real life, children should never boss their parents around. In programming, I would argue children shouldn't even know who their parents are!

In 8 Patterns to Help You Destroy Massive View Controller, I suggested a Navigator type that can be injected into view controllers that contains the logic for moving through your app. Navigators are a fine solution if you need it in one place, but we quickly run into an issue that navigators can't help us with.

Those three lines have a lot of responsibilities in it, but that one view controller isn't the only place that this is happening. Imagine you have a photo editing app.

Your PhotoSelectionViewController presents your StraighteningViewController which presents your FilteringViewController which presents your CaptioningViewController. Your navigation flow is now spread among three different objects. Further, something is presenting your PhotoSelectionViewController, but the dismissal has to be handled in CaptioningViewController.

Passing a Navigator around keeps these view controllers all coupled together in a chain, and doesn't really solve the problem of each view controller knowing about the next one in the chain.

Coordinators will help us solve this problem.

Libraries vs Frameworks

I think Apple expects us to write code in all these ways. They want us to make the view controller the center of the world, because apps written all in the same style let them make the most impact with their SDK changes. Unfortunately, for developers, that's not always the best move for us. We're the ones who are responsible for maintaining our apps into the future, and dependable design and malleability of code are much higher priorities for us.

They say the distinction between libraries and frameworks is that you call libraries, and frameworks call you. I want to treat 3rd-party dependencies as much like libraries as possible.

When using UIKit, you're not in charge. You call -pushViewController:animated: and it does a bunch of work and at some indeterminate time in the future, it calls -viewDidLoad: on the next view controller, where you can do some more stuff. Rather than let UIKit decide when your code runs, you should get out of UIKit-land as soon as possible, so you can have full control over how your code flows.

I used to think of view controllers as the highest level thing in app, the things that know how to run the whole show. But I started wondering what it might look like to flip that around. A view is transparent to its view controller. It's bossed around by its view controller. What if we made the view controller just another thing that's transparent in the same way?


What is a coordinator?

So what is a coordinator? A coordinator is an object that bosses one or more view controllers around. Taking all of the driving logic out of your view controllers, and moving that stuff one layer up is gonna make your life a lot more awesome.

It all starts from the app coordinator. The app coordinator solves the problem of the overstuffed app delegate. The app delegate can hold on to the app coordinator and start it up. The app coordinator will set up the primary view controller for the app. You'll find can find this pattern in the literature, in books like Patterns of Enterprise Application Architecture. They call it the Application Controller. The app coordinator is a special version of an application controller, specifically made for iOS. The app coordinator can create and configure view controllers, or it can spawn new child coordinators to perform subtasks.

Which responsiblities do coordinators take over from the view controller? Primarily, navigation and model mutation. (By model mutation I mean saving the user's changes to the database, or making a PUT or POST request to an API, anything that can destructively modify the user's data.)

When we take those tasks out of a view controller, we end up with a view controller that's inert. It can be presented, it can fetch data, transform it for presentation, display it, but it crucially can't alter it. We know now that any time we present a view controller, it won't take things into its own hands. Whenever it needs to let us know about an event or user input, it uses a delegate method. Let's take a look at a code example.

Code Example

Let's start from the app delegate.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
    self.window = [[UIWindow alloc] initWithFrame:[UIScreen mainScreen].bounds];
    self.rootViewController = [[UINavigationController alloc] init];

    self.appCoordinator = [[SKAppCoordinator alloc] initWithNavigationController:self.rootViewController];
    [self.appCoordinator start];

    [self.window makeKeyAndVisible];

The app delegate sets up the apps window and root view controller, and then fires up the app coordinator. The coordinator's initialization is separated from starting its work. This lets us create it however we want (lazily, greedily, etc), and only start it when we're ready.

The coordinator is just a simple NSObject:

@interface SKAppCoordinator : NSObject

This is great. There's no secrets going on here. The UIViewController class is thousands of lines, and we don't know what exactly will happen when we call any of its methods, because it's closed-source. Making the objects that run our app simple NSObject types makes everything simpler.

The app coordinator initialized with the data it needs, which includes the root view controller.

- (instancetype)initWithNavigationController:(UINavigationController *)navigationController {
    self = [super init];
    if (!self) return nil;

    _navigationController = navigationController;

    return self;

Once we hit the -start method, the coordinator sets off to work.

- (void)start {
    if ([self isLoggedIn]) {
        [self showContent];
    } else {
        [self showAuthentication];

Right from the start, coordinator is making decisions. Previously, logic like this didn't have a home. You could maybe stick it in a view controller or in the app delegate, but those both have their flaws. In a view controller, you have one view controller which is making some decisions well-beyond its purpose, or you're polluting the app delegate with stuff it doesn't care about.

Let's examine the -showAuthentication method. Here, our base coordinator spawns off child coordinators to do work and subtasks.

- (void)showAuthentication {
    SKAuthenticationCoordinator *authCoordinator = [[SKKAuthenticationCoordinator alloc] initWithNavigationViewController:self.navigationController];
    authCoordinator.delegate = self;
    [authCoordinator start];
    [self.childCoordinators addObject:authCoordinator];

We use an array of childCoordinators to prevent the child coordinators from getting deallocated.

View controllers exist in a tree, and each view controller has a view. Views exist in a tree of subviews, and each subview has a layer. Layers exist in a tree. Becuase of this childCoordinators array, you get a tree of coordinators.

This child coordinator will create some view controllers, wait on them to do work, and signal us when it's done. When a coordinator signals that its finished, it cleans itself up, popping off whatever view controllers it has added, and then uses delegates to get messages back up to its parent.

Once we've authenticated, we'll get a delegate message, and, we can allow the child coordinator to be deallocated, and then we can get back to our regularly-scheduled programming.

- (void)coordinatorDidAuthenticate:(SKAuthenticationCoordinator *)coordinator {
    [self.childCoordinators removeObject:coordinator];
    [self showContent];

Inside the authentication coordinator, it creates whatever view controllers it needs, pushes them onto the navigation controller. Let's take a look at what that looks like.

@implementation AuthCoordinator

- (instancetype)initWithNavigationController:(UINavigationController *)navigationController {
    self = [super init];
    if (!self) return nil;

    _navigationController = navigationController;

    return self;

Initialization is similar to the app coordinator.

- (void)start {
    SKFirstRunViewController *firstRunViewcontroller = [SKFirstRunViewController new];
    firstRunViewcontroller.delegate = self;
    [self.navigationController pushViewController:firstRunViewcontroller animated:NO];

Authentication needs to start with a "first run view controller". This view controller has buttons for sign up and log in and maybe a little slideshow explaining the app. Let's push that view controller on and become its delegate.

This view controller has a delegate so we can be informed when the user taps the "sign up" button. Instead of the view controller needing to know which signup view controller to create and present, the coordinator will handle that.

- (void)firstRunViewControllerDidTapSignup:(SKFirstRunViewController *)firstRunViewController {
    SKSignUpViewController *signUpViewController = [[SKSignUpViewController alloc] init];
    signupViewController.delegate = self;
    [self.navigationController pushViewController:signupViewController animated:YES];

We become the sign up view controller's delegate so that it can inform us when its buttons are pushed.

- (void)signUpViewController:(SKSignUpViewController *)signupViewController didTapSignupWithEmail:(NSString *)email password:(NSString *)password {

And so on. Here we actually perform the work of the signup API request and saving the authentication token, and then we inform our parent coordinator.

Whenever anything happens with a view controller (like user input) the view controller will tell its delegate (in this case the coordinator) and the coordinator will execute the actually task that the user intended. It's important to have the coordinator do the work, so that the view controller remains inert.

Why are coordinators great?

  1. Each view controller is now isolated.
    View controllers don't know anything beyond how to present their data. Whenever anything happens, it tells its delegate, but of course it doesn't know who its delegate is.

    Before, when there was a fork in the road, the view controller needs to ask "Okay, well, am I on the iPad or the iPhone?". "Is the user being A/B tested?" They no longer have to ask any questions like that. Have we merely pushed this question, this conditional, up to the coordinator? In a way, but we can solve it in a much better way up there.

    When we do need to have two flows at once, for A/B testing or multiple size classes, you can just swap the entire coordinator object instead of sticking a bunch of conditionals all over your view controllers.

    If you want to understand the way a flow works, that's now super easy, since all the code is in one place.

  2. View controllers are now reusable.
    They don't assume anything about what context they'll be presented in, or what their buttons will be used for. They can be used and reused for their good looks, without dragging any logic along with them.

    If you're writing your iPad version of your app, the only thing you need to replace are your coordinators, and you can reuse all the view controllers.

  3. Every task and sub-task in your app now has a dedicated way of being encapsulated.
    Even if the task works across multiple view controllers, it's encapsulated. If your iPad version reuses some of those subtasks but not others, it's really easy to use just those sub-tasks.

  4. Coordinators separate display-binding from side effects.
    You never again have to worry about if a view controller will mess up your data when you present a view controller. It can only read and display, never write or corrupt data. This is a similar concept to command-query separation.

  5. Coordinators are objects fully in your control.
    You're not sitting around waiting for -viewDidLoad to get called so you can do work, you're totally in control of the show. There's no invisible code in a UIViewController superclass that is doing some magic that you don't understand. Instead of being called, you start doing the calling.

    Flipping this model makes it much easier to understand what's going on. The behavior of your app is a completely transparent to you, and UIKit is now just a library that you call when you want to use it.

The current project I'm working will be open-sourced in a few weeks, and it uses coordinators to manage all of its view controllers. The app coordinator and auth coordinator examples come from that project.

Ultimately, coordinators are just an organizational pattern. There's no library you can use for coordinators because they're so simple. There's no pod you can install and nothing to subclass from. There's not even really a protocol to conform to. Rather than being a weakness, this is a strength of using a pattern like coordinators: it's just your code, with no dependencies.

They'll help make your app and your code more manageable. View controllers will be more reusable, and growing your app will be easier than ever.

September 28, 2015

What The Heck Is A Monad

There's an everpresent hilarious thread in programming blogs where an author tries to explain what a monad is, starts strong, and then ends up losing everyone with some nonsense about an endofunctor.

I'm going to take a crack at it. I'm probably going to fail, and I'm going to prove Soroush's Burrito Law in the process: anyone trying to explain monads is going to fail, even if they account for how hard explaining monads is.

A monad is a wrapper for a thing. (It honestly is like a burrito.)

The Maybe Type

So. Let's talk Swift. Let's say you have a function that reads something from disk. It can either return the thing, or it can return nothing. Swift calls this type Optional, but let's recreate it and call it Maybe.

enum Maybe<WrappedType> {
    case Something(WrappedType)
    case Nothing

Angle-bracket blindness is real. WrappedType here just means that our Maybe can have anything inside of it, and we can refer to whatever type that thing is with WrappedType. This lets the compiler know the type of what's coming out is the same as the type of what's put in. Let's continue. Imagine a function called readString() reads a string from the disk, where its provenance is doubtful. It "maybe" doesn't exist.

func readString() -> Maybe<String> {
    return .Something("someString");

let maybeString = readString()

This is the first important part of a monad. You have to have a way to create one. In this case, the constructor, Maybe.Something, fills that role. In other languages, this is known as unit or the inconveniently-named function return. It's a function that takes one parameter, and returns a monad that wraps that parameter.

If this is all we have, it's kind of frustrating to use. To get access to the information inside that Maybe, you have to explicitly unwrap it. The structure of the type forces you to check it, every time.

switch (maybeString) {
    case let .Something(definitelyString):
    case .Nothing:

After we use Maybe with a case statement like this for a while, we notice that we're writing this switch statement over and over. It's a lot of boilerplate just so we can access the Something.

Maybe we can wrap up the switch into a function, and pass that function a block that will be executed if the Maybe is a Something and not a Nothing.

extension Maybe {
    func ifSomething<NewWrappedType>(block: (WrappedType) -> Maybe<NewWrappedType>) -> Maybe<NewWrappedType> {
        switch self {
        case let .Something(wrapped):
            return block(wrapped)
        case .Nothing:
            return .Nothing

I would love to remove the types in the function declaration to make it shorter and clearer, but they actually confer a very important piece of information: The block that we're accepting takes a WrappedType (a string, in the example above), and returns a new type wrapped in a monad.

This is very important. ifSomething doesn't just give us access to the Something, it also lets us transform it, but requires that it be wrapped in the Monad type again. (Sometimes we want to transform but not wrap it, and I'll address that in a moment.)

It's important that the block returns an already-wrapped monad, so that we can chain these calls. This is a big part of why monads are useful.

Now that we have this power of transforming the wrapped thing, we can do something very cool. Imagine we had a function that converts strings to JSON objects. It returns a Maybe, since deserializing JSON can fail.

readDataFromDisk().ifSomething({ string in
    return convertToJSON(string)

or, more succinctly:


At the end of .ifSomething(convertToJSON), we just get back a new Maybe monad, on which we can again call ifSomething. This is how monads let us chain stuff. Imagine another function called getProperty, which also returns a Maybe.

readDataFromDisk().ifSomething({ string in
    return convertToJSON(string)
}).ifSomething({ JSON in
    return getProperty(JSON, "username")

Et cetera. We can keep chaining like this, as long as we need to. Notice how this is flat, instead of nested. We've totally skipped the part where you have to unwrap an Optional with an if let, then unwrap another one, then another one, and you end up with something 5 levels deep.

If we didn't have ifSomething, our code would look like this:

var maybeData = readDataFromDisk()
if let data = maybeData {
    let maybeJSON = convertToJSON(string)
    if let JSON = maybeJSON {
        let maybeUsername = getProperty(JSON, "username")
        if let username = maybeUsername {
            //we can finally use the username

While this might work in early stages, the fact that it continues to indent inwards means that it's not scalable. With monads, we can remove the nesting and organize our code much better.

Functional programmers took a great name like ifSomething and made it totally inscrutable by calling it flatMap. (In some of the literature, it's also known as bind. In Haskell, aka peak inscrutability, it's invoked with the operator >>=.)

bind (or flatMap) and unit (the constructor) are all it takes to be considered a monad. From those two, we can also build map. map lets us transform the wrapped object without having to rewrap it ourselves at the end of the function. This is particularly useful for arrays and other collection types.

To build map, we wrap the result of the map block with the constructor and send that to flatMap:

extension Maybe {
    func map<NewWrappedType>(block: (WrappedType) -> NewWrappedType) -> Maybe<NewWrappedType> {
        return flatMap({ wrapped in
            return .Something(block(wrapped));

In this way, map can be written in terms of bind and unit.

The Monadic Laws

For something to be monad, in addition to implementing bind and unit, it has to follow some special rules.

First, left identity.

unit(a).flatMap(f) == f(a)

Wrapping a in the monad, then calling flatMap with any function f will have same result as just calling f with a, since f returns a new monad.

Second, right identity.

m.flatMap(unit) == m

Since the unit function doesn't do anything but take the unwrapped value and wrap it, calling flatMap on an existing monad m with unit will have no effect.

The first two monadic laws exist to assert that unit, the constructor, doesn't do anything other than wrap a.

Lastly, associativity.

m.flatMap(f).flatMap(g) == m.flatMap({ a in
    return f(a).flatMap(g)

This says that we can combine two functions f and g into a new function, and calling flatMap with the new function is the same as calling flatMap with each of the functions separately.

In Swift

You don't need to write your own Maybe type to start using this in Swift today. Swift's Optional type supports flatMap.

let optionalString = Optional.Some("123");
let optionalInt = optionalString.flatMap { string in
    return Int(string)

Because all versions of flatMap behave the same, you can use this just like the flatMap we wrote above, chaining it over and over.

In A Nutshell

That's monads in a nutshell. Well, they are the nutshell. They wrap a value, like a shell.

Other monads you might have seen include:

  • Result, which wraps a "Something" with an optional "Error".
  • Eventually/Promise/Deferred, which wraps a value that doesn't exist yet.
  • Array, which wraps many values.

With Result in particular, it's easy to see how you might have a series of functions where each is dependent on the previous one, and where each can fail and generate an error. You would have a pyramid of doom without being able to flatMap them repeatedly.

I've used the Promise monad a lot in Javascript, and it organizes code greatly:

User.signup = function(user) {
    return new Promise(function(resolve, reject) {
        User.validate(user).then(function() {
            return Bcrypt.genSalt(10);
        }).then(function(salt) {
            return Bcrypt.hash(user.password, salt, null);
        }).then(function(saltedHashedPassword) {
            return User.insertIntoDatabase(user, saltedHashedPassword);
        }).then(function(userRecord) {
        }).catch(function(error) {

The chaining that the promise monad affords us is crucial when all of your operations are asynchronous and can fail. We return a new Promise in each then block, and the chain continues. In this way, we've described an inherently complex, asynchronous task in a serial list of steps.

Monads are weird thing. The idea lets us treat all these different wrappers, which all serve different functions, similarly. When we know that something is a monad, we gain a ton of knowledge about how we can use it and what it can do.

After all, they're just monoids in the category of endofunctors.

September 24, 2015

The Limited Subset

Two of the most impressive languages in my mind are Lisp and Smalltalk. They are both extremely limited: Lisp is structured around spaces and parentheses. That's it.

Smalltalk has only 6 keywords: true, false, nil, self, super, and thisContext. Everything in Smalltalk, including conditionals, is handled by sending messages to objects. How many keywords does Swift have already? Why can't I name my UISwitch "switch"? (I know why, and it's vexing.)

Most languages don't do this. Their users want more toys and more sugar in the language. The language maintainers add it, and you end up with a languages that are a complicated mess.

While most langauges aren't designed with simplicity as a feature, I've found that with a little self-control, it's possible to intentionally limit yourself to a subset of your favorite language, and have great results.

I'm writing Objective-C and Javascript on a day-to-day basis, so my examples will come from those languages. The idea, however, is universalizable.

First, app structure. When structuring an app, I limit myself to one structure. Inasmuch as I can control it, everything has one and only one owner, so that my object graph looks as much like a tree as possible. From the top, the app is run by coordinators, which boss around the view controllers. Those view controllers each have a view, which is a custom subclass of UIView designed specifically for each view controller. This lets me push subview configuration and layout out of the view controller and down into the view. Subview allocation is always handled via lazy loading. Layout itself is pushed even further into a layout object.

I'll break these rules every once in a blue moon, but almost every time, I will write code like this. It helps increase consistency and removes any doubt about how to structure the simple stuff. When a new programmer comes on board, if they understand one section of the app, they'll be able to easily navigate through the rest of it.

Second, communication patterns. Since everything is a tree, messages need to go down, from parents to children, and up, from children to their parents. Parents know about their children, so they can just send simple messages. But when children talk to parents, I almost always use the delegate pattern exclusively. I know about KVO, I know about NSNotificationCenter, and I know about the responder chain, but delegates have a simplicity to them that I value. Their relationships are one-to-one (each object generally has one delegate), their data is typed, and it's easy to search for delegate conformance. There's a small cost to the extra code of declaring the protocol, but I find that it pays off in the long run.

Delegates are great, but the key here is not the pattern itself, but limiting yourself to it. The artifical limitation simplifies decision-making while writing code. When you limit yourself intentionally, you don't end with a tangled mess of code that fires an NSNotification from inside of block that calls another method via a delegate.

Finally, a Javascript example. The project I've been working on has a backend written in Express. Express assumes very little about your code's structure. It gives you the standard Node.js Request and Response types, as well as a Router. That's pretty much it. Routers let you register functions with routes. Now, I could try to build a roughly object-oriented model layer within Javascript, but I don't really understand how Javascript constructors work, so that choice is out.

Instead of trying to make each route (represented by a function) talk to an object-oriented model layer, the User type holds onto all of the things that a user can do.

var loginAction = function(req, res, next) {
    var creds = { email: req.body.email, password: req.body.password };
    User.login(creds).then(function(session) {
        res.sendJSON({ session: session });
    }).catch(function(error) {
        res.sendError(error, "There was an error logging in.");

Each of the things on the User type is a function, and calling it returns a new Promise. This, and the fact that data is represented by dumb objects that do nothing but hold the data, make the model layer is very functional in nature. Of course, it wouldn't be interesting if it were only in one place, so every single route in the system works like this.

When I'm making a new route, it's no longer necessary to think about how go from the functional "action" layer to the Promise-based model layer. It's always the same. The sendJSON and sendError functions on Result are also almost always used, making it dead simple to add new endpoints. You get to the meat of writing your code even quicker. Limiting you and providing a One True Way to do stuff is how frameworks like Rails help you speed up development.

When you limit yourself to a small part of the language, you end up with code that's simpler and requires less thought to implement. It removes a lot of the weight from working in your codebase and makes it easier for outsiders to understand how things work.

September 10, 2015

Learning Lessons The Hard Way

This happens more often than I’d care to admit. While programming, I’ll receive a flash of insight that maps cleanly onto a trite phrase I’d heard dozens of times before. Prefer composition to inheritance. The law of Demeter. Code is read a lot more than it is written. The pith is endless.

You’ll come across a problem and realize how doing it the easy way has bitten you in the ass. You’ll stop making the mistake of doing it the easy way, and start doing it the right way. I’ve found that with lessons like these, you almost always have to go the long way around, experiencing them firsthand before there can be a deep understanding.

A great example of one of these lessons is making small classes and small methods. Ben Orenstein gives advice to this effect in one of his posts. I remember defending to my Ruby-writing coworkers: maybe you could write Ruby that way, but Objective-C is just too complex and verbose to write with one-line methods. I was, of course, super wrong. Not only is it possible to write Objective-C in that style, it’s also joyful.

David Foster Wallace, in his now-classic commencement address for Kenyon College, said:

Because a huge percentage of the stuff that I tend to be automatically certain of is, it turns out, totally wrong and deluded. I have learned this the hard way, as I predict you graduates will, too.

There’s a delightfully meta component here: not only are the lessons hard to learn, but the lesson that the lessons are hard to learn is hard to learn.

There’s a somber component too: learning lessons the hard way sucks. Our knowledge is stuck in our brains, and can only come out in low-bandwidth forms, like prose, speech, or drawing.

When you finally have that bug that causes you to have the epiphany, all you end up with is a shitty platitude. You have no way to impart to anyone else what you’ve discovered. The lesson took time for you to learn, and once you’ve learned it, you wish you could give it to other people. If you’ve ever written a monad tutorial, I’m looking at you.

I’ve found only two ways to route around it. The first is to feel someone else’s suffering at the hands of one of these bugs and experience empathy.

The everpresent joke in computer science says there’s two hard problems: naming and cache invalidation. When I was starting out, I remember thinking “ha ha, there’s no way cache invalidation could be that hard”. Fast forward a few years, and while I’ve learned a few lessons of my own about bad caches, reading Code Climate’s incident report for a major issue made me understand on a far deeper level what it means for cache invalidation to be hard. Lesson learned, bon mot deployed.

The second approach to teaching hard ideas is to change the value function. Programmers take shortcuts and write code “the bad way” because it’s way easier. It feels as though the cost is lower than the hard way, because you can’t see the interest acquired by your technical debt. Instead of making the cost of shortcut higher, we can increase the value of the “right way”.

To make the hard way more appealing to your audience, the trick is finding the perfect example to illustrate it. Most readers will skim a new post when it comes out. They don’t stop and pick out each word you write. Code blocks break up the writing, and they provide somewhere for the readers eye to jump. Given that you’re only getting a small amount of the reader’s attention, you have to find an example that strikes the right note with the reader. The example decides if the post lands or not. It has to be relatable and short, so that you don’t lose the attention of the reader, but it has to be complex enough to not be trivial.

For example, I’m very proud of my post on the method object pattern, Graduation. This post makes it simple, obvious, and elegant to take a method and make a new class out of it. It readjusts the value function in the programmer. When the value of the good way increases, it takes far less activation energy to jump from the bad way to the good way. Put another way, the intensity of the lesson you have to learn is much lower before you’re happy to switch over.

Part of why I love Sandi Metz’s work is that she takes a simple problem, like inheritance, and thinks about it a lot, and finds the right words to make even the most novice programmer understand why to prefer composition to inheritance. I’d heard about this platitude plenty when I was just starting out, but Sandi’s talk was what really drove it home.

Ultimately, we can only do so much. It’s a shame that these methods have to be learned over and over, the slow way. The best we can do is grease that path for those that follow us.

September 1, 2015

Many Models

When you first start making an app, your choices about the model layer will impact you the most in the long run. It'll pay dividends to think about these choices up front, instead of being left with the detritus of accidental decisions.

One of these decisions is how to cache your data: whether you'll use Core Data or something simpler.

Another big decision is whether each unique model is represented by one instance or many. This is, in some ways, the crux of the difference between object-oriented and functional/immutable styles. More simply put, it's the class way versus the struct way. Let's examine the differences.

With the object-oriented style, you have living, breathing models. You can send messages to the model, and it can respond, make decisions, perform network requests, and generally act as a first-class citizen in your app.

Making this work requires a design pattern called the Identity Map, which is just a big dictionary that maps each object instance to its identifier. When fetching a model from a store (whether it's a network store or a store persisted on the device), each instance is checked against the Identity Map. Objective-C's flexible initialization makes this really easy.

- (instancetype)initWithObjectID:(id<NSCopying, NSObject>)objectID {
    id existingObject = [[SKModel identityMap] objectForKey:objectID];
    if (existingObject) {
        self = existingObject
        return existingObject;

    self = [super init];
    if (!self) return nil;

    self.objectID = objectID;
    [[SKModel identityMap] setObject:self forKey:objectID];

    return self;

Core Data does this for you. If you fetch the same object twice (on the same object context), you will get the same instance back. From the Core Data Documentation:

"Core Data ensures that—in a given managed object context—an entry in a persistent store is associated with only one managed object. The technique is known as uniquing. Without uniquing, you might end up with a context maintaining more than one object to represent a given record."

Given that your stores will return the one instance for each object ID, that model can change out from under any controller-type objects that are holding on to it. Therefore, each controller needs to observe changes on its models and re-render its view to reflect those changes.

There are two cases in which this approach works really well. The first case is when there are many views on screen, some of which point to the same models. Making a change in one view should also be reflected in the other views representing the same object. It also is great with data persisted on-disk. Data on-disk changes frequently and nonatomically. For a todo app, the user might change the due date, which would save the model to disk, then the priority, which would save the model again. Using the same model object makes our program simpler.

The second approach is to use many instances for each individual model. For this approach, each time you fetch from your store (again, either network or persisted), you create a fresh struct (or struct-like object) and use that. When modifying, either ensure isolation of each object by fetching it anew in each place that you'll need it, or by using copy-on-write to create a new instance for each modification.

This approach shines on single-screen platforms, like iOS, where the user is generally looking at one thing at a time. In cases like this, you can lazily refresh data when it comes back on screen, rather than refreshing it greedily. It also shines in systems where the "source of truth" is on a server. Any mutating REST call is an atomic change that will return a response that is fully-formed and fully-validated by the server application. It's also great for immutable data, like tweets. When things can't be edited by the user, it's much safer to use a system that prefers less mutation, like structs.

While using actual Swift structs grants some guarantees about how the thing will be used, it comes with some cost as well. Drew Crawford writes about the "Structs Philosophy™".

The insight here is that doing anything of value involves calling at least one mutating function and as soon as you do that, your function must be mutating, and everything that calls you must be mutating, and it's mutating all the way up, except for the silly leaf nodes like CGRect or your bag of Ints.

Like Drew, I'm not sure I can advise making your whole model layer out of purely Swift structs. For one, any kind of mutation is costly, especially for deeply nested heirarchies of data. Second, structs can't really represent members like images, data, colors, and URLs as value types yet, even though they are often components of models and clearly are values. Using those types requires bridging to Objective-C, which loses a lot of guarantees of immutability and isolation. Lastly, it requires making your models somewhat "dumb". While you can attach functions to structs in Swift, they seem to be more for manipulating the data rather than doing any work, like making an API request that regards the model.

The choice between many instances and one is in your hands. Don't make the decision lightly, however. The path you choose will affect the bedrock of your app, and it will be hard to change later.

August 26, 2015

The Scrivener

There's a special class of scribe that takes requirements from a client and translates them into highly precise, sometimes arcane documents. Few laypeople can understand these documents and the policies they encode, but they nevertheless have a great effect on everyday life.

The domain of the clients and requirements are often different than as the domain of the scribe; the scribe has to learn to efficiently translate the rules of the business she represents into the precise language of the documents.

When these documents are parsed and analyzed, they're done by another body. The meaning drawn from the documents won't necessarily be the same as what the scribe or her clients intended, but the effects of the documents stand, nevertheless.

These scribes are paid a lot of money for their skills, and a lot of people find it frustrating that no one else can do what they do. But ultimately, it's not really possible have a modern, functioning society without them.

I'm talking about lawyers writing contracts, of course.

The parallels between lawyers and programmers are myriad. These parallels are sometimes trivial, like defining variables; every programmer has seen an employment contract that has text like "Widgets LLC, hereinafter referred to as 'the Company'". Defining a variable in this way shortens the contract, prevents redefinition errors, and makes the contract easier to change.

Lawyers also have to flesh out simple ideas (for example, the statement "things that I make") into really complex and precise statements ("all Inventions that I may solely or jointly author, discover, develop, conceive, or reduce to practice during the period of the Relationship").

Contracts are more similar to interpreted code rather than compiled code. The contract can be "statically analyzed" (i.e., read by other lawyers) while it's being written, but its meaning won't be fully determined until it's challenged and taken before a judge.

Judges are much more lenient than code interpreters, so typos and other simple errors aren't commonly held against a lawyer the way they might be with a programmer. Ambiguity, however, provides ammo that opposing counsel can use to change the intent of the contract. That's a runtime error if I've ever heard of one! If only we could give lawyers automated tests.

Much fuss is made about the future of programming, whether it will be text, or some kind of semantic editor, or perhaps something graphical. On the other hand, the future of contracts appears to be a lot more form contracts and contract generation, rather than a dramatic rethinking of their representation. Generated, parameterized contracts parallel either functions or libraries in the programming, although it strains the metaphor a little bit.

It's also easy to imagine the profession of programming going in the same direction as lawyering. Extreme demand and limited supply has caused engineer salaries to skyrocket to the around (and above, in some cases) a lawyer's starting salary. The long hours and benefits (free food if you stay past a certain time!) are also slowly starting resemble big law firms. Programming culture could definitely use the specialized trade schools and a the strict pipeline that law benefits from. Our lawyerly brothers and sisters have already been through what we're going through, and we've got a lot to learn from them.

August 18, 2015

The Back of the Fence

When I write code, my goal is to take as few shortcuts as possible. People often ask me why I bother.

I bother because shortcuts join together like Voltron to tangle your code. I bother because it’s hard enough to read my code when it’s written well. I bother because I never know who’s going to be looking at my code.

Code quality is precisely the proverbial “back of the fence”. Perhaps apocryphally, Steve Jobs was known to care about small, sometimes invisible details. He would fuss over the beauty in a circuit’s design, because his father inspired him to consider the minutiae:

It was important, his father said, to craft the backs of cabinets and fences properly, even though they were hidden. “He loved doing things right. He even cared about the look of the parts you couldn’t see.”

It applies to more than just code and circuit boards. I’ve noticed the best designers that I’ve worked with have meticulously assembled PSD files. Deep hierarchies of organization, consistently-named layers, edges perfectly between pixels rather than on top of them. This attention to detail is reflected in the quality of the designs as well; truly, someone who sweats the details sweats the big picture, too.

The back of the fence never aligns with business metrics; code quality is no exception. It’s a second-order effect which can only affect your bottom line in indirect ways. It’s cheaper to write the code right the first time, rather than having to fix its bugs later. It’s cheaper to work with supple code, code that’s been designed with change in mind.

(There’s one little hack. If you’re working on an open-source project that’ll be used by developers, then your code quality is no longer merely adjacent to cost. You can effortlessly align your business metrics and code.)

In some cases, it’s not possible to draw even the most tenuous connection between the concerns of your business and back-of-the-fence style code quality. For those times, we might call it professional pride.

Joe Cieplinski is a designer who (not by coincidence, I’m sure!) creates extremely neat PSDs. I’ll leave you with some remarks from Joe’s talk at CocoaLove last year:

We don’t design beautiful things hoping that people notice. We design beautiful things knowing that they probably won’t. […] We do design for us. We do design because we want to sleep at night.

August 12, 2015

Bend the Language

Swift brings lots of awesome new features. I'm looking forward to using lots of them, even though it's still a bit early for me to adopt the language. Even if you don't want to or can't adopt the new language yet, you should still be able to get your hands on the new features that they've created, like value types, richer enums, and protocol extensions. In some cases, you might even want to be able to experiment and get access to these features even before Apple announces them.

Luckily for us, a language that’s rich enough lets us approximate these features. In the immortal words of Kanye West, "everything in the world is exactly the same." Let's take a look at a few ways how we can get Swifty features in Objective-C.

Value Types

The first awesome Swift feature we want to examine is the Swift struct and its value semantics. To know how to replicate this feature in something like Objective-C, we need to first figure out how we use it and which parts of the feature we want.

Unfortunately, developing this distinction between inert values and living, breathing instances isn't easy. Via the Swift book:

As a general guideline, consider creating a structure when one or more of these conditions apply:

  • The structure’s primary purpose is to encapsulate a few relatively simple data values.
  • It is reasonable to expect that the encapsulated values will be copied rather than referenced when you assign or pass around an instance of that structure.
  • Any properties stored by the structure are themselves value types, which would also be expected to be copied rather than referenced.
  • The structure does not need to inherit properties or behavior from another existing type.

My general criterion is if you don't care which copy of a thing you have, it's a value.

So, given that description of values, we primarily want to take advantage of their immutability and isolation. (There are some slight performance gains as well, but come on — profile before optimizing.)

For a mutable thing to cause unintentional bugs in your app, it must also be accessed from more than one place; after all, if it’s only every used in one place, it can’t change out from under you. Structs in Swift solve this problem by either being immutable, in which case you can’t change them, or being copy-on-mutate: as soon as you change one, you get a fresh copy that’s yours and yours alone.

To get the same benefit, we need to either preserve either the immutability of our object or its isolation. To preserve isolation in Objective-C, we can conform to <NSCopying> and declare all properties as copy. This is pretty tedious and it’s easy to forget to copy every time you use the object in a new place.

Immutability, on the other hand, is all defined in the class’s interface, and lets us give the class's users hints about how the class should be used.

In The Value of Value Objects, I write about using a pattern called Tiny Types (gist) to get immutable wrappers around values.

@interface Hostname : ValueObject

- (instancetype)initWithString:(NSString *)name;

@property (readonly) NSString *name;
@property (readonly) NSURL *hostnameAsURL;
@property (readonly) Hostname *hostnameByEnsuringSSL;


@implementation Hostname

- (instancetype)initWithString:(NSString *)name {
    return [self initWithBackingObject:name];

- (NSString *)name {
    return self.backingObject;

- (NSURL *)hostnameAsURL {
    return [NSURL URLWithString:self.name];

- (Hostname *)hostnameByEnsuringSSL {
    NSURLComponents *URLComponents = [NSURLComponents componentsWithURL:URL resolvingAgainstBaseURL:YES];
    URLComponents.scheme = @"https";
    return [[Hostname alloc] initWithString:URLComponents.URL.absoluteString];


Because this object isn't mutable, you never have to worry about it changing out from underneath you. Even when it does need to be changed, as in -hostnameByEnsuringSSL, it returns a new Hostname object, never changing the existing one.

In the same way that Swift's compiler enforces proper behavior, so too does Objective-C's. With a message like -hostnameByEnsuringSSL, the name and its type signature make it clear that something different is happening, and that you need to handle it in a special way.

This pattern can be extended to allow intialization with multiple "backing objects" rather than just one, but the principles (immutable properties and copy-on-mutate) stay the same.

Rich Enumerations

Swift's enums are a great step forward from C's enumerations. The greatest advantage is that you can associate both data and functions with the enum now. In the old C-style, enumerations were nothing more than some sugar around a number:

typedef enum Shape : NSUInteger {
} Shape;

To do work with these, you have to pass them into a top-level or free function:

NSInteger calculateArea(Shape s);

Reversing the subject and object like this is harder to read, for some reason, and no one likes it.

Swift allows us to describe create an enum more easily, associate data with it (such as the radius or width and height below). Once we have data, we can add behavior as well:

enum Shape {
    case Circle(Double)
    case Rect(Double, Double)

    func area() -> Double {
        switch self {
        case let Circle(r):
            return pi * r * r
        case let Rect(width, height):
            return width * height

Ultimately, though, these enumerations are all "sum types", or tagged unions. They can be in one of many disjoint states. Fortunately for us, sum types can take many forms, and we have access to some of them in Objective-C. If we want a sum type with associated data and functions, just like Swift’s, we can easily get that.

@interface Shape : NSObject

@property NSInteger area;


@implementation Circle : Shape

- (instancetype)initWithRadius:(NSinteger)radius {
    self = [super init];
    self.radius = radius;
    return self;

- (NSInteger)area {
    return M_PI * self.radius * self.radius;


Here, instead of an enum called Shape, we have an abstract class called Shape, and several concrete classes that derive from it, defining their own initializers and bringing their own implementations for -area.

There’s a caveat here, which is that this solution uses inheritance. However, this is something I'd call "inheritance with intention". It's small, contained, and out of the way. The polymorphism defined early in the process, allowing it to be designed holistically. It lets other objects operate with less knowledge of how Shape works and minimizes fewer code paths. If it still bothers you, you can get the same "sum type" effect without inheritance by using protocols.

This pattern is discussion with more depth in Replace Enumerations With Polymorphism.

Protocol Extensions

Another big awesome feature in Swift is protocol extensions. I love this one and I feel like I've wanted it in Objective-C forever. At its core, protocol extensions add extra behavior to an abstract set of messages. I've been leaning very heavily on decoration to get the same thing (behavior added to a preexisting thing) when I'm in Objective-C.

Let's look at an example. A data source object helps you map index paths to objects, so that they can be displayed in a table view or collection view. A protocol for a data source might be defined like so:

@protocol DataSourceProtocol

@property NSInteger numberOfSections;
- (NSInteger)numberOfObjectsInSection:(NSInteger)section;
- (id)objectAtIndexPath:(NSIndexPath *)indexPath;


This is the bare minimum we need to implement this protocol. If we were in Swift and we wanted to add any functions whose results were derived from these, we could use protocol extensions. In Objective-C, however, we'll use decoration:

@implementation FullDataSource

- (instancetype)initWithDataSource:(id<DataSourceProtocol>)dataSource { //... }

//forward -numberOfSections, -numberOfObjectsInSection: and -objectAtIndexPath: to self.dataSource

- (NSArray *)allObjects {
    NSMutableArray *allObjects = [NSMutableArray array];
    NSInteger numberOfSections = self.numberOfSections;
    for (NSInteger sectionIndex = 0; sectionIndex < numberOfSections; sectionIndex++) {
        NSInteger numberOfObjectsInSection = [self numberOfObjectsInSection:sectionIndex];
        for (NSInteger objectIndex = 0; objectIndex < numberOfObjectsInSection; objectIndex++) {
            [allObjects addObject:[self objectAtIndexPath:[NSIndexPath indexPathForRow:objectIndex inSection:sectionIndex]]];
    return allObjects;

- (NSIndexPath *)indexPathForObject:(id)object {
    for (NSInteger sectionIndex = 0; sectionIndex < self.numberOfSections; sectionIndex++) {
        NSInteger numberOfObjectsInSection = [self numberOfObjectsInSection:sectionIndex];
        for (NSInteger objectIndex = 0; objectIndex < numberOfObjectsInSection; objectIndex++) {
            id object = [self objectAtIndexPath:[NSIndexPath indexPathForRow:objectIndex inSection:sectionIndex];
            if ([object isEqual:object]) {
                return object;
    return nil;


By wrapping <DataSourceProtocol> in DataSource, we can add behavior to any data source defined with that protocol, like -allObjects and a reverse lookup method called -indexPathForObject:. We could also add a method to enumerate over the objects with a block, and so on.

Whereas with the enumeration example we had caveats, here we have two major advantages. First, decoration can be done multiple times. If you want to wrap a wrapped thing, you can do that. Second, you gain the ability to change which decorators you are using dynamically. Some <DataSourceProtocol> objects need some decorators, and others need different ones. You can mix and match them however you like.


I've written about all these patterns here with more depth: value objects here, enumerations here, and decoration here, here, here, and here. Yes, Swift allows you to express these ideas much more tersely, and in some cases, prevents you from making errors while expressing them. This is to Swift's benefit. These ideas are more flexible than their specific implementations in Swift, however. As you implement them, you get to decide exactly how they work.

It's also notable that the blog posts describing these patterns came out before their respective Swift counterparts. If you’re willing to bend the traditional uses of the language a little, and you can reap their benefits, and you can do it before everyone else. Don't wait for anyone. Write your own future.

August 6, 2015

There are no mysteries

I've been programming in earnest for about 5 years. When I was just beginning, a lot of the way the computers and programming worked seemed like sheer magic. As I've gotten better at programming, a lot of that opacity has started to fade away. The illusions of abstraction are disappearing. It's a little terrifying that the people who make the foundation of our technology aren't that much smarter than me or you, but mostly this epiphany overjoys me. Nothing is inscrutable!

When I first started making iOS apps, I remember wiring up my IBAction to a button in Interface Builder and thinking "Okay, but how is this method getting called?" If I'm remembering right, part of me thought that it might be some kind of polling loop. I also remember thinking that it couldn't possibly be that stupid. Turns out, it is. NSRunLoop is just a big while loop that spins and makes your whole app go around.

Arrays are easy enough; they're just an unbroken block of memory. With an index for an item, you can figure out what its memory location is: (offset + index*item_size). But how the heck do dictionaries (or hashes, or maps) work? How can they get instant access with a key that isn't a number?

You've got to convert that key into a number, so you can put it into an array. (Turns out, this is one of the things you learn when you get a real computer science degree in a class called "Data Structures". I had a degree-having friend explain it to me.) Take a big array, hash the key into a number (hence, the name "hash"), mod it by the size of the array so it'll fit into one of the array's slots, and insert it into a bucket at that index. When fetching, the key is hashed again, and it should fall into the same spot in the array, and you can get the item from the bucket. It's not magic, even though it certainly feels like it. Here's a good Mike Ash post with code explaining how NSMutableDictionary is written.

The big kahuna: compilers. How do they work? You could go read the daunting Structure and Interpretation of Computer Programs (which I promise I will do one of these days), or read this relatively short (16,000-word) blog post that explains how to write a lexer, parser, and evaluator for small subset of Lisp. It took me about two hours to get through, but in the end I had a pretty good sense for how Lisp (probably the simplest programming language) was parsed and compiled. How do you make this Lisp compile down to machine code? I don't know. Yet.

Richard Feynman, when asked if understanding our world via science removes its magic, responds with a parable about an artist:

He'll hold up a flower and say "look how beautiful it is," and I'll agree. Then he says "I as an artist can see how beautiful this is but you as a scientist take this all apart and it becomes a dull thing," and I think that he's kind of nutty. […] All kinds of interesting questions which the science only adds to the excitement, the mystery and the awe of a flower. It only adds. I don't understand how it subtracts.

Garbage collectors; NSCache; image convolution; UIScrollView; bloom filters: with all of it, there's no voodoo. Even if it feels like a black box that you'll never be able to peer into, it's all just code. No matter how many stupid arcane meaningless equations are on the Wikipedia page, it's simpler than that.

It's all just bits and bytes that are arranged in nice ways that have nice ramifications. You can understand those bits and bytes, and knowing that the computing world has order is essential to understanding that you can affect it and improve it.

July 29, 2015

The Braid of Thought

Warning: This one’s not about programming, but it is programming adjacent.

Meditations on Moloch, which I’ve linked to before, is a great article. Alexander finds a few texts and weaves them together, creating an argument that that binds them all. He wraps each text into that braid until an free-standing argument is borne from each of the separate texts.

Chronology is a harsh master. You read three totally unrelated things at the same time and they start seeming like obviously connected blind-man-and-elephant style groping at different aspects of the same fiendishly-hard-to-express point.

A braid of thought — multiple ideas that come together from varied sources that you happened upon almost by chance — follows.

As We May Think is a classic article from 1945, where Vannevar Bush describes a piece of technology he calls the “memex”.

A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.

Upon reading this, many humans rejoice, seeing a reflection of the memex in the modern web or in Wikipedia. Bush seemed to have predicted the widespread dissemination of knowledge that's taken for granted today. But Bret Victor, when reading Bush’s piece, sees failure in the modern web. He writes about this failure in the Web of Alexandria and its follow up.

The web, of course, took a different approach. A million volumes, yes, but our desks remain empty. Instead, when we summon a volume, we are granted a transient and ephemeral peek at its sole instance, out there somewhere in the world, typically secured within a large institution.

This server/client, truth-lives-in-the-cloud, single-point-of-failure model is so engrained in me that when considering a new product, I don’t even evaluate other protocols for data storage and transmission. But there’s so many other templates we can base our information model off of.

Consider email, where every participant keeps a copy of every single dispatch. Consider Git, where each programmer keeps a copy of every single commit. Consider Bittorrent, where each user hosts only the files they care about.

There are also models we haven’t tried. Some datasets are really small, like my contacts. There’s no reason I couldn’t trust all those contacts to 5 or 6 of my close friends. I doubt they’d mind a few hundred extra kilobytes on their drives. Even If I didn’t trust them, I can just heavily encrypt the contacts before I send them over.

It’s made me realize that there’s so many ways we limit ourselves with technology. Modern startups care about very specific things, like streams and attention and keeping your data. Silicon Valley’s conception of what an app can be is very narrow minded, bounded by the dreams of hockey-stick user growth and a high valuation. Paul Ford explains why these types of companies want that type of data in his post about Ashley Madison.

I’ve never built a translucent database-driven system because none of my clients have ever been the least bit interested. They want names, addresses, credit cards, and the like. But they don’t actually need a lot of that data to build a good web service. They need it for potential marketing purposes.

These connections are yet reinforced by Maciej Cegłowski’s Web Design: The First 100 Years. The connections here are left as an exercise for the reader.

If I’m understanding Victor’s argument correctly, it’s the very structure of the web (combined with a thirst for profit, I would probably add) makes these problems arise. We could restructure things and make the web suck less by default. A pit of success of usability and humanity.

To cap off this little mini-web of interconnectivity, last week Mike Caulfield wrote about taking this idea further in Beyond Conversation. He described how links fit into this world: “Links are made by readers as well as writers,” and that was the moment for me that all these threads wound themselves into a much stronger braid.

We have values: links shouldn’t rot; users should have control of their data; media companies should serve users, and not the other way around. These values are incompatible with the Internet in its current conception, and we can’t build the future we want on top of a foundation that won’t support it.

Caulfield’s general solution is for each user to create her own wiki. A personal wiki has never had much appeal for me, since I don’t have a category of stuff I write that I wouldn't publish here. I don’t really write much privately. However, you could take all the pages I love, all the pages I think are important, all the pages I think are mildly interesting, all the pages I’ve seen, all the conversations I’ve had, and all the pictures I’ve taken and save them on my computer. Make it searchable. Now you’re talking about something I really understand.

Allow me to make links and create associations on top of these documents and this very blog post becomes a lot easier to research and write. Chat logs that have links in them should point to the pages that they link to; those pages should link back to the chat logs. What if there were two documents I loved that both linked to a third document on the web? I’d probably want to read that. What if documents, web pages, and images could cluster around a physical location? I’d probably want to know when I’m near that spot.

It’s hard to solve the link rot problem in the general case, without downloading the whole Internet. Fortunately, I don’t care about most of the Internet; I only care about the stuff I’ve read. We have so much cheap storage available now that it’s almost criminal not to save all the web pages you look at. But we don’t keep them so someone can sell ads to you more easily; we keep them so you can easily find the stuff you liked and cared about and thought about.

The idea of the “outboard brain” is a technique for using computers for what computers are good at, and freeing up your brain for what brains are good at. This tool is an outboard brain for idea generation. ("An enlarged intimate supplement to his memory"!) I think I remember all of the various articles and writings that lead to this blog post, but what if I haven’t? Maybe I’ve forgotten a thread that would pull this braid in a totally different direction. (Oh, yeah. I just remembered: watching the BBC’s Connections definitely put me in the mood to start thinking about how all these things are related. – ed.) I’m really bad at remembering stuff, and I’d love to relegate that responsibility to a tool that’s great at it.

Computers are for people. Let's make them so.