johnezang/JSONKit

JSONKit

JSONKit is dual licensed under either the terms of the BSD License, or alternatively under the terms of the Apache License, Version 2.0.
Copyright © 2011, John Engelhart.

A Very High Performance Objective-C JSON Library

UPDATE: (2011/12/18) The benchmarks below were performed before Apples NSJSONSerialization was available (as of Mac OS X 10.7 and iOS 5). The obvious question is: Which is faster, NSJSONSerialization or JSONKit? According to this site, JSONKit is faster than NSJSONSerialization. Some quick "back of the envelope" calculations using the numbers reported, JSONKit appears to be approximately 25% to 40% faster than NSJSONSerialization, which is pretty significant.

Parsing	Serializing
23% Faster than Binary .plist !	549% Faster than Binary .plist !

• Benchmarking was performed on a MacBook Pro with a 2.66GHz Core 2.
• All JSON libraries were compiled with gcc-4.2 -DNS_BLOCK_ASSERTIONS -O3 -arch x86_64.
• Timing results are the average of 1,000 iterations of the user + system time reported by getrusage.
• The JSON used was twitter_public_timeline.json from samsoffes / json-benchmarks.
• Since the .plist format does not support serializing NSNull, the null values in the original JSON were changed to "null".
• The experimental branch contains the gzip compression changes.
  • JSONKit automagically links to libz.dylib on the fly at run time– no manual linking required.
  • Parsing / deserializing will automagically decompress a buffer if it detects a gzip signature header.
  • You can compress / gzip the serialized JSON by passing JKSerializeOptionCompress to -JSONDataWithOptions:error:.

JSON versus PLIST, the Ultimate Showdown benchmarks the common JSON libraries and compares them to Apples .plist format.

JavaScript Object Notation, or JSON, is a lightweight, text-based, serialization format for structured data that is used by many web-based services and API's. It is defined by RFC 4627.

JSON provides the following primitive types:

• null
• Boolean true and false
• Number
• String
• Array
• Object (a.k.a. Associative Arrays, Key / Value Hash Tables, Maps, Dictionaries, etc.)

These primitive types are mapped to the following Objective-C Foundation classes:

JSONKit uses Core Foundation internally, and it is assumed that Core Foundation ≡ Foundation for every equivalent base type, i.e. CFString ≡ NSString.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

JSON To Objective-C Primitive Mapping Details

• The JSON specification is somewhat ambiguous about the details and requirements when it comes to Unicode, and it does not specify how Unicode issues and errors should be handled. Most of the ambiguity stems from the interpretation and scope RFC 4627 Section 3, Encoding: JSON text SHALL be encoded in Unicode. It is the authors opinion and interpretation that the language of RFC 4627 requires that a JSON implementation MUST follow the requirements specified in the Unicode Standard, and in particular the Conformance chapter of the Unicode Standard, which specifies requirements related to handling, interpreting, and manipulating Unicode text.

  The default behavior for JSONKit is strict RFC 4627 conformance. It is the authors opinion and interpretation that RFC 4627 requires JSON to be encoded in Unicode, and therefore JSON that is not legal Unicode as defined by the Unicode Standard is invalid JSON. Therefore, JSONKit will not accept JSON that contains ill-formed Unicode. The default, strict behavior implies that the JKParseOptionLooseUnicode option is not enabled.

  When the JKParseOptionLooseUnicode option is enabled, JSONKit follows the specifications and recommendations given in The Unicode 6.0 standard, Chapter 3 - Conformance, section 3.9 Unicode Encoding Forms. As a general rule of thumb, the Unicode code point U+FFFD is substituted for any ill-formed Unicode encountered. JSONKit attempts to follow the recommended Best Practice for Using U+FFFD: Replace each maximal subpart of an ill-formed subsequence by a single U+FFFD.

  The following Unicode code points are treated as ill-formed Unicode, and if JKParseOptionLooseUnicode is enabled, cause U+FFFD to be substituted in their place:

  U+0000.
  U+D800 thru U+DFFF, inclusive.
  U+FDD0 thru U+FDEF, inclusive.
  U+nFFFE and U+nFFFF, where n is from 0x0 to 0x10

  The code points U+FDD0 thru U+FDEF, U+nFFFE, and U+nFFFF (where n is from 0x0 to 0x10), are defined as Noncharacters by the Unicode standard and "should never be interchanged".

  An exception is made for the code point U+0000, which is legal Unicode. The reason for this is that this particular code point is used by C string handling code to specify the end of the string, and any such string handling code will incorrectly stop processing a string at the point where U+0000 occurs. Although reasonable people may have different opinions on this point, it is the authors considered opinion that the risks of permitting JSON Strings that contain U+0000 outweigh the benefits. One of the risks in allowing U+0000 to appear unaltered in a string is that it has the potential to create security problems by subtly altering the semantics of the string which can then be exploited by a malicious attacker. This is similar to the issue of arbitrarily deleting characters from Unicode text.

  RFC 4627 allows for these limitations under section 4, Parsers: An implementation may set limits on the length and character contents of strings. While the Unicode Standard permits the mutation of the original JSON (i.e., substituting U+FFFD for ill-formed Unicode), RFC 4627 is silent on this issue. It is the authors opinion and interpretation that RFC 4627, section 3 – Encoding, JSON text SHALL be encoded in Unicode. implies that such mutations are not only permitted but MUST be expected by any strictly conforming RFC 4627 JSON implementation. The author feels obligated to note that this opinion and interpretation may not be shared by others and, in fact, may be a minority opinion and interpretation. You should be aware that any mutation of the original JSON may subtly alter its semantics and, as a result, may have security related implications for anything that consumes the final result.

  It is important to note that JSONKit will not delete characters from the JSON being parsed as this is a requirement specified by the Unicode Standard. Additional information can be found in the Unicode Security FAQ and Unicode Technical Report #36 - Unicode Security Consideration, in particular the section on non-visual security.

• The JSON specification does not specify the details or requirements for JSON String values, other than such strings must consist of Unicode code points, nor does it specify how errors should be handled. While JSONKit makes an effort (subject to the reasonable caveats above regarding Unicode) to preserve the parsed JSON String exactly, it can not guarantee that NSString will preserve the exact Unicode semantics of the original JSON String.

  JSONKit does not perform any form of Unicode Normalization on the parsed JSON Strings, but can not make any guarantees that the NSString class will not perform Unicode Normalization on the parsed JSON String used to instantiate the NSString object. The NSString class may place additional restrictions or otherwise transform the JSON String in such a way so that the JSON String is not bijective with the instantiated NSString object. In other words, JSONKit can not guarantee that when you round trip a JSON String to a NSString and then back to a JSON String that the two JSON Strings will be exactly the same, even though in practice they are. For clarity, "exactly" in this case means bit for bit identical. JSONKit can not even guarantee that the two JSON Strings will be Unicode equivalent, even though in practice they will be and would be the most likely cause for the two round tripped JSON Strings to no longer be bit for bit identical.

• JSONKit maps true and false to the CFBoolean values kCFBooleanTrue and kCFBooleanFalse, respectively. Conceptually, CFBoolean values can be thought of, and treated as, NSNumber class objects. The benefit to using CFBoolean is that true and false JSON values can be round trip deserialized and serialized without conversion or promotion to a NSNumber with a value of 0 or 1.

• The JSON specification does not specify the details or requirements for JSON Number values, nor does it specify how errors due to conversion should be handled. In general, JSONKit will not accept JSON that contains JSON Number values that it can not convert with out error or loss of precision.

  For non-floating-point numbers (i.e., JSON Number values that do not include a . or e|E), JSONKit uses a 64-bit C primitive type internally, regardless of whether the target architecture is 32-bit or 64-bit. For unsigned values (i.e., those that do not begin with a -), this allows for values up to 264-1 and up to -263 for negative values. As a special case, the JSON Number -0 is treated as a floating-point number since the underlying floating-point primitive type is capable of representing a negative zero, whereas the underlying twos-complement non-floating-point primitive type can not. JSON that contains Number values that exceed these limits will fail to parse and optionally return a NSError object. The functions strtoll() and strtoull() are used to perform the conversions.

  The C double primitive type, or IEEE 754 Double 64-bit floating-point, is used to represent floating-point JSON Number values. JSON that contains floating-point Number values that can not be represented as a double (i.e., due to over or underflow) will fail to parse and optionally return a NSError object. The function strtod() is used to perform the conversion. Note that the JSON standard does not allow for infinities or NaN (Not a Number). The conversion and manipulation of floating-point values is non-trivial. Unfortunately, RFC 4627 is silent on how such details should be handled. You should not depend on or expect that when a floating-point value is round tripped that it will have the same textual representation or even compare equal. This is true even when JSONKit is used as both the parser and creator of the JSON, let alone when transferring JSON between different systems and implementations.

• For JSON Objects (or NSDictionary in JSONKit nomenclature), RFC 4627 says The names within an object SHOULD be unique (note: name is a key in JSONKit nomenclature). At this time the JSONKit behavior is undefined for JSON that contains names within an object that are not unique. However, JSONKit currently tries to follow a "the last key / value pair parsed is the one chosen" policy. This behavior is not finalized and should not be depended on.

  The previously covered limitations regarding JSON Strings have important consequences for JSON Objects since JSON Strings are used as the key. The JSON specification does not specify the details or requirements for JSON Strings used as keys in JSON Objects, specifically what it means for two keys to compare equal. Unfortunately, because RFC 4627 states JSON text SHALL be encoded in Unicode., this means that one must use the Unicode Standard to interpret the JSON, and the Unicode Standard allows for strings that are encoded with different Unicode Code Points to "compare equal". JSONKit uses NSString exclusively to manage the parsed JSON Strings. Because NSString uses Unicode as its basis, there exists the possibility that NSString may subtly and silently convert the Unicode Code Points contained in the original JSON String in to a Unicode equivalent string. Due to this, the JSONKit behavior for JSON Strings used as keys in JSON Objects that may be Unicode equivalent but not binary equivalent is undefined.

  See also:
      W3C - Requirements for String Identity Matching and String Indexing

Objective-C To JSON Primitive Mapping Details

• When serializing, the top level container, and all of its children, are required to be strictly invariant during enumeration. This property is used to make certain optimizations, such as if a particular object has already been serialized, the result of the previous serialized UTF8 string can be reused (i.e., the UTF8 string of the previous serialization can simply be copied instead of performing all the serialization work again). While this is probably only of interest to those who are doing extraordinarily unusual things with the run-time or custom classes inheriting from the classes that JSONKit will serialize (i.e, a custom object whose value mutates each time it receives a message requesting its value for serialization), it also covers the case where any of the objects to be serialized are mutated during enumeration (i.e., mutated by another thread). The number of times JSONKit will request an objects value is non-deterministic, from a minimum of once up to the number of times it appears in the serialized JSON– therefore an object MUST NOT depend on receiving a message requesting its value each time it appears in the serialized output. The behavior is undefined if these requirements are violated.

• The objects to be serialized MUST be acyclic. If the objects to be serialized contain circular references the behavior is undefined. For example,

  [arrayOne addObject:arrayTwo];
  [arrayTwo addObject:arrayOne];
  id json = [arrayOne JSONString];
  

  … will result in undefined behavior.

• The contents of NSString objects are encoded as UTF8 and added to the serialized JSON. JSONKit assumes that NSString produces well-formed UTF8 Unicode and does no additional validation of the conversion. When JKSerializeOptionEscapeUnicode is enabled, JSONKit will encode Unicode code points that can be encoded as a single UTF16 code unit as \uXXXX, and will encode Unicode code points that require UTF16 surrogate pairs as \uhigh\ulow. While JSONKit makes every effort to serialize the contents of a NSString object exactly, modulo any RFC 4627 requirements, the NSString class uses the Unicode Standard as its basis for representing strings. You should be aware that the Unicode Standard defines string equivalence in a such a way that two strings that compare equal are not required to be bit for bit identical. Therefore there exists the possibility that NSString may mutate a string in such a way that it is Unicode equivalent, but not bit for bit identical with the original string.

• The NSDictionary class allows for any object, which can be of any class, to be used as a key. JSON, however, only permits Strings to be used as keys. Therefore JSONKit will fail with an error if it encounters a NSDictionary that contains keys that are not NSString objects during serialization. More specifically, the keys must return YES when sent -isKindOfClass:[NSString class].

• JSONKit will fail with an error if it encounters an object that is not a NSNull, NSNumber, NSString, NSArray, or NSDictionary class object during serialization. More specifically, JSONKit will fail with an error if it encounters an object where -isKindOfClass: returns NO for all of the previously mentioned classes.

• JSON does not allow for Numbers that are ±Infinity or ±NaN. Therefore JSONKit will fail with an error if it encounters a NSNumber that contains such a value during serialization.

• Objects created with [NSNumber numberWithBool:YES] and [NSNumber numberWithBool:NO] will be mapped to the JSON values of true and false, respectively. More specifically, an object must have pointer equality with kCFBooleanTrue or kCFBooleanFalse (i.e., if((id)object == (id)kCFBooleanTrue)) in order to be mapped to the JSON values true and false.

• NSNumber objects that are not booleans (as defined above) are sent -objCType to determine what type of C primitive type they represent. Those that respond with a type from the set cislq are treated as long long, those that respond with a type from the set CISLQB are treated as unsigned long long, and those that respond with a type from the set fd are treated as double. NSNumber objects that respond with a type not in the set of types mentioned will cause JSONKit to fail with an error.

  More specifically, CFNumberGetValue(object, kCFNumberLongLongType, &longLong) is used to retrieve the value of both the signed and unsigned integer values, and the type reported by -objCType is used to determine whether the result is signed or unsigned. For floating-point values, CFNumberGetValue is used to retrieve the value using kCFNumberDoubleType for the type argument.

  Floating-point numbers are converted to their decimal representation using the printf format conversion specifier %.17g. Theoretically this allows up to a float, or IEEE 754 Single 32-bit floating-point, worth of precision to be represented. This means that for practical purposes, double values are converted to float values with the associated loss of precision. The RFC 4627 standard is silent on how floating-point numbers should be dealt with and the author has found that real world JSON implementations vary wildly in how they handle this issue. Furthermore, the %g format conversion specifier may convert floating-point values that can be exactly represented as an integer to a textual representation that does not include a . or e– essentially silently promoting a floating-point value to an integer value (i.e, 5.0 becomes 5). Because of these and many other issues surrounding the conversion and manipulation of floating-point values, you should not expect or depend on floating point values to maintain their full precision, or when round tripped, to compare equal.

Reporting Bugs

Please use the github.com JSONKit Issue Tracker to report bugs.

The author requests that you do not file a bug report with JSONKit regarding problems reported by the clang static analyzer unless you first manually verify that it is an actual, bona-fide problem with JSONKit and, if appropriate, is not "legal" C code as defined by the C99 language specification. If the clang static analyzer is reporting a problem with JSONKit that is not an actual, bona-fide problem and is perfectly legal code as defined by the C99 language specification, then the appropriate place to file a bug report or complaint is with the developers of the clang static analyzer.

Important Details

• JSONKit is not designed to be used with the Mac OS X Garbage Collection. The behavior of JSONKit when compiled with -fobjc-gc is undefined. It is extremely unlikely that Mac OS X Garbage Collection will ever be supported.

• JSONKit is not designed to be used with Objective-C Automatic Reference Counting (ARC). The behavior of JSONKit when compiled with -fobjc-arc is undefined. The behavior of JSONKit compiled without ARC mixed with code that has been compiled with ARC is normatively undefined since at this time no analysis has been done to understand if this configuration is safe to use. At this time, there are no plans to support ARC in JSONKit. Although tenative, it is extremely unlikely that ARC will ever be supported, for many of the same reasons that Mac OS X Garbage Collection is not supported.

• The JSON to be parsed by JSONKit MUST be encoded as Unicode. In the unlikely event you end up with JSON that is not encoded as Unicode, you must first convert the JSON to Unicode, preferably as UTF8. One way to accomplish this is with the NSString methods -initWithBytes:length:encoding: and -initWithData:encoding:.

• Internally, the low level parsing engine uses UTF8 exclusively. The JSONDecoder method -objectWithData: takes a NSData object as its argument and it is assumed that the raw bytes contained in the NSData is UTF8 encoded, otherwise the behavior is undefined.

• It is not safe to use the same instantiated JSONDecoder object from multiple threads at the same time. If you wish to share a JSONDecoder between threads, you are responsible for adding mutex barriers to ensure that only one thread is decoding JSON using the shared JSONDecoder object at a time.

Tips for speed

• Enable the NS_BLOCK_ASSERTIONS pre-processor flag. JSONKit makes heavy use of NSCParameterAssert() internally to ensure that various arguments, variables, and other state contains only legal and expected values. If an assertion check fails it causes a run time exception that will normally cause a program to terminate. These checks and assertions come with a price: they take time to execute and do not contribute to the work being performed. It is perfectly safe to enable NS_BLOCK_ASSERTIONS as JSONKit always performs checks that are required for correct operation. The checks performed with NSCParameterAssert() are completely optional and are meant to be used in "debug" builds where extra integrity checks are usually desired. While your mileage may vary, the author has found that adding -DNS_BLOCK_ASSERTIONS to an -O2 optimization setting can generally result in an approximate 7-12% increase in performance.

• Since the very low level parsing engine works exclusively with UTF8 byte streams, anything that is not already encoded as UTF8 must first be converted to UTF8. While JSONKit provides additions to the NSString class which allows you to conveniently convert JSON contained in a NSString, this convenience does come with a price. JSONKit must allocate an autoreleased NSMutableData large enough to hold the strings UTF8 conversion and then convert the string to UTF8 before it can begin parsing. This takes both memory and time. Once the parsing has finished, JSONKit sets the autoreleased NSMutableData length to 0, which allows the NSMutableData to release the memory. This helps to minimize the amount of memory that is in use but unavailable until the autorelease pool pops. Therefore, if speed and memory usage are a priority, you should avoid using the NSString convenience methods if possible.

• If you are receiving JSON data from a web server, and you are able to determine that the raw bytes returned by the web server is JSON encoded as UTF8, you should use the JSONDecoder method -objectWithUTF8String:length: which immediately begins parsing the pointers bytes. In practice, every JSONKit method that converts JSON to an Objective-C object eventually calls this method to perform the conversion.

• If you are using one of the various ways provided by the NSURL family of classes to receive JSON results from a web server, which typically return the results in the form of a NSData object, and you are able to determine that the raw bytes contained in the NSData are encoded as UTF8, then you should use either the JSONDecoder method objectWithData: or the NSData method -objectFromJSONData. If are going to be converting a lot of JSON, the better choice is to instantiate a JSONDecoder object once and use the same instantiated object to perform all your conversions. This has two benefits:

  1. The NSData method -objectFromJSONData creates an autoreleased JSONDecoder object to perform the one time conversion. By instantiating a JSONDecoder object once and using the objectWithData: method repeatedly, you can avoid this overhead.
  2. The instantiated object cache from the previous JSON conversion is reused. This can result in both better performance and a reduction in memory usage if the JSON your are converting is very similar. A typical example might be if you are converting JSON at periodic intervals that consists of real time status updates.

• On average, the JSONData… methods are nearly four times faster than the JSONString… methods when serializing a NSDictionary or NSArray to JSON. The difference in speed is due entirely to the instantiation overhead of NSString.

• If at all possible, use NSData instead of NSString methods when processing JSON. This avoids the sometimes significant conversion overhead that NSString performs in order to provide an object oriented interface for its contents. For many uses, using NSString is not needed and results in wasted effort– for example, using JSONKit to serialize a NSDictionary or NSArray to a NSString. This NSString is then passed to a method that sends the JSON to a web server, and this invariably requires converting the NSString to NSData before it can be sent. In this case, serializing the collection object directly to NSData would avoid the unnecessary conversions to and from a NSString object.

Parsing Interface

JSONDecoder Interface

The objectWith… methods return immutable collection objects and their respective mutableObjectWith… methods return mutable collection objects.

Note: The bytes contained in a NSData object MUST be UTF8 encoded.

Important: Methods will raise NSInvalidArgumentException if parseOptionFlags is not valid.
Important: objectWithUTF8String: and mutableObjectWithUTF8String: will raise NSInvalidArgumentException if string is NULL.
Important: objectWithData: and mutableObjectWithData: will raise NSInvalidArgumentException if jsonData is NULL.

+ (id)decoder;
+ (id)decoderWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)initWithParseOptions:(JKParseOptionFlags)parseOptionFlags;

- (void)clearCache;

- (id)objectWithUTF8String:(const unsigned char *)string length:(size_t)length;
- (id)objectWithUTF8String:(const unsigned char *)string length:(size_t)length error:(NSError **)error;
- (id)mutableObjectWithUTF8String:(const unsigned char *)string length:(size_t)length;
- (id)mutableObjectWithUTF8String:(const unsigned char *)string length:(size_t)length error:(NSError **)error;

- (id)objectWithData:(NSData *)jsonData;
- (id)objectWithData:(NSData *)jsonData error:(NSError **)error;
- (id)mutableObjectWithData:(NSData *)jsonData;
- (id)mutableObjectWithData:(NSData *)jsonData error:(NSError **)error;

NSString Interface

- (id)objectFromJSONString;
- (id)objectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)objectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;
- (id)mutableObjectFromJSONString;
- (id)mutableObjectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)mutableObjectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;

NSData Interface

- (id)objectFromJSONData;
- (id)objectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)objectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;
- (id)mutableObjectFromJSONData;
- (id)mutableObjectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)mutableObjectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;

JKParseOptionFlags

Serializing Interface

The serializing interface includes NSString convenience methods for those that need to serialize a single NSString. For those that need this functionality, the NSString additions are much more convenient than having to wrap a single NSString in a NSArray, which then requires stripping the unneeded […] characters from the serialized JSON result. When serializing a single NSString, you can control whether or not the serialized JSON result is surrounded by quotation marks using the includeQuotes: argument:

Note: The NSString methods that do not include a includeQuotes: argument behave as if invoked with includeQuotes:YES.
Note: The bytes contained in the returned NSData object are UTF8 encoded.

NSArray and NSDictionary Interface

- (NSData *)JSONData;
- (NSData *)JSONDataWithOptions:(JKSerializeOptionFlags)serializeOptions error:(NSError **)error;
- (NSString *)JSONString;
- (NSString *)JSONStringWithOptions:(JKSerializeOptionFlags)serializeOptions error:(NSError **)error;

NSString Interface

- (NSData *)JSONData;
- (NSData *)JSONDataWithOptions:(JKSerializeOptionFlags)serializeOptions includeQuotes:(BOOL)includeQuotes error:(NSError **)error;
- (NSString *)JSONString;
- (NSString *)JSONStringWithOptions:(JKSerializeOptionFlags)serializeOptions includeQuotes:(BOOL)includeQuotes error:(NSError **)error;

JKSerializeOptionFlags