Khanlou

Vapor’s JSON handling leaves something to be desired. Pulling something out of the request’s JSON body looks like this:

var numberOfSpots: Int?

init(request: Request) {
	self.numberOfSpots = request.json?["numberOfSpots"]?.int
}

There’s a lot of things I don’t like about this code.

1. The int hanging off the end is extraneous: The type system already knows self.numberOfSpots is an int; ideally, I wouldn’t have to tell it twice.

2. The optional situation is out of control. The JSON property might not exist on the request, the title property might not exist in the JSON, and the property might not be a int. Each of those branches is represented by an Optional, which is flattened using the optional chaining operator. At the end of the expression, if the value is nil, there’s no way to know which one of the components failed.

3. At the end of the ridiculous optional chain, the resulting numberOfSpots value must be optional. If I need the numberOfSpots property to be required, I need to add an extra variable and a guard.

    guard let numberOfSpots = request.json?["numberOfSpots"]?.int else {
        throw Abort(status: .badRequest, reason: "Missing 'numberOfSpots'.")
    }
    self.numberOfSpots = numberOfSpots
   

   Needless to say, this is bad code made worse. The body of the error doesn’t contain any information besides the key “numberOfSpots”, so there’s a little more duplication there, and that error isn’t even accurate in many cases. If the json property of the request is nil, that means that either the Content-Type header was wrong or that the JSON failed to parse, neither of which are communicated by the message “Missing ‘numberOfSpots’.” If the “numberOfSpots” key was present, but stored a string (instead of an int), the .int conversion would fail, resulting in an optional, and the error message would be equally useless.

Probably more than half of the requests in the Beacon API have JSON bodies to parse and dig values out of, so this is an important thing to get right.

The broad approach here is to follow the general model for how we parse JSON on the client. We can use type inference to deal with the extraneous conversions, and errors instead of optionals.

Let’s look at the errors first. We’ve discussed three possible errors: missing JSON, missing keys, and mismatched types. Perfect for an error enum:

enum JSONError: AbortError {

	var status: Status {
		return .badRequest
	}
	
	case missingJSON
	case missingKey(keyName: String)
	case mismatchedType(keyName: String, expectedType: String, actualType: String)
	
	var reason: String {
		switch self {
		case .missingJSON:
			return "The endpoint requires a JSON body and a \"Content-Type\" of \"application/json\"."
		case let .missingKey(keyName):
			return "This endpoint expects the JSON key \(missingKey), but it wasn't present."
		case let .mismatchedType(keyName, expectedType, actualType):
			return "This endpoint expects the JSON key '\(key)'. It was present, but did not have the expected type \(expectedType). It had type '\(actualType).'"
		}
	}
}

Once the errors have been laid out, we can begin work on the rest of this implementation. Using a similar technique to the one laid out Decoding JSON in Swift, we can begin to build things up. (I call it NiceJSON because Vapor provides a json property on the Request, and I’d like to not collide with that.)

class NiceJSON {
	let json: JSON?

	public init(json: JSON?) {
		self.json = json
	}

	public func fetch<T>(_ key: String) throws -> T {
		// ...
	}

}

However, here, we run into the next roadblock. I typically store JSON on the client as a [String: Any] dictionary. In Vapor, it’s stored as a StructuredData, which is an enum that can be stored in one of many cases: .number, .string, .object, .bool, .date, and so on.

While this is strictly more type-safe (a JSON object can’t store any values that aren’t representable in one of those basic forms — even though .date and .data are a cases of StructuredData, ignore them for now), it stands in the way of this technique. You need a way to bridge between compile-time types (like T and Int), and run-time types (like knowing to call the computed property .string). One way to handle this is to check the type of T precisely.

public func fetch<T>(_ key: String) throws -> T {

	guard let json = self.json else { throw JSONError.missingJSON }
	
	guard let untypedValue = json[key] else { throw JSONError.missingKey(key) }
	
	if T.self == Int.self {
		guard let value = untypedValue.int else {
			throw JSONError.mismatchedType(key, String.self)
		}
		return value
	}
	// handle bools, strings, arrays, and objects
}

While this works, it has one quality that I’m not crazy about. When you access the .int computed property, if the case’s associated value isn’t a int but can be coerced into a int, it will be. For example, if the consumer of the API passes the string “5”, it’ll be silently converted into a number. (Strings have it even worse: numbers are converted into strings, boolean values become the strings "true" and "false", and so on. You can see the code that I’m referring to here.)

I want the typing to be a little stricter. If the consumer of the API passes me a number and I want a string, that should be a .mismatchedType error. To accomplish this, we need to destructure Vapor’s JSON into a [String: Any] dictionary. Digging around the vapor/json repo a little, we can find code that lets us do this. It’s unfortunately marked as internal, so you have to copy it into your project.

extension Node {
	var backDoorToRealValues: Any {
		return self.wrapped.backDoorToRealValues
	}
}

extension StructuredData {
	internal var backDoorToRealValues: Any {
		switch self {
		case .array(let values):
			return values.map { $0.backDoorToRealValues }
		case .bool(let value):
			return value
		case .bytes(let bytes):
			return bytes
		case .null:
			return NSNull()
		case .number(let number):
			switch number {
			case .double(let value):
				return value
			case .int(let value):
				return value
			case .uint(let value):
				return value
			}
		case .object(let values):
			var dictionary: [String: Any] = [:]
			for (key, value) in values {
				dictionary[key] = value.backDoorToRealValues
			}
			return dictionary
		case .string(let value):
			return value
		case .date(let value):
			return value
		}
	}
}

The code is pretty boring, but it essentially converts from Vapor’s JSON type to a less well-typed (but easier to work with) object. Now that we have this, we can write our fetch method:

var dictionary: [String: Any]? {
	return json?.wrapped.backDoorToRealValues as? [String: Any]
}

public func fetch<T>(_ key: String) throws -> T {
	guard let dictionary = dictionary else {
		throw JSONError.jsonMissing()
	}
	
	guard let fetched = dictionary[key] else {
		throw JSONError.missingKey(key)
	}
	
	guard let typed = fetched as? T else {
		throw JSONError.typeMismatch(key: key, expectedType: String(describing: T.self), actualType: String(describing: type(of: fetched)))
	}
	
	return typed
}

It’s pretty straightforward to write implementations of fetchOptional(_:), fetch(_, transformation:), and the other necessary functions. I’ve gone into detail on them in the Decoding JSON in Swift post and in the Parser repo for that post, so I won’t dwell on those implementations here.

For the final piece, we need a way to access our new NiceJSON on a request. For that, I added a computed property to the request:

extension Request {
    public var niceJSON: NiceJSON {
        return NiceJSON(json: json)
    }
}

This version of the code creates a new NiceJSON each time you access the property, which can be optimized a little bit by constructing it once and sticking it in the storage property of the Request.

Finally, we can write the nice code that we want at the call site.

var numberOfSpots: Int

init(request: Request) throws {
	self.numberOfSpots = try request.niceJSON.fetch("numberOfSpots")
}

This code has provides a non-optional value, no duplication, and generates descriptive errors.

There’s on last gotcha that I want to go over: numbers in JSON. As of Vapor 2, all JSON numbers are stored as Double values. This means fetching numbers will only work if you fetch them as a Double. This doesn’t appear to be documented anywhere, so I’m not sure how much we should rely on it, but it appears to currently work this way. I think the reason for it is NSNumber subtyping weirdness. On Mac Foundation, numbers in JSON are stored in the NSNumber type, which can be as casted to a Double, Bool, Int, or UInt. Because of the different runtime, that stuff doesn’t work the same way in Linux Foundation, so everything is stored in the least common denominator format, Double, which can (more or less) represent Int and UInt types.

I have a small workaround for this, which at the top of the fetch method, add a special case for Int:

public func fetch<T>(_ key: String) throws -> T {
    if T.self == Int.self {
        return try Int(self.fetch(key) as Double) as! T
    }

Bool works correctly without having to be caught in a special way, and you shouldn’t use UInt types regularly in your code. The note is copied below for posterity and link rot:

Use UInt only when you specifically need an unsigned integer type with the same size as the platform’s native word size. If this isn’t the case, Int is preferred, even when the values to be stored are known to be nonnegative. A consistent use of Int for integer values aids code interoperability, avoids the need to convert between different number types, and matches integer type inference, as described in Type Safety and Type Inference.

NiceJSON is a small abstraction that lets you work with JSON in clean way, without having to litter your code with guards, expectations, and hand-written errors about the presence of various keys in the JSON body.