((section 2 "Outdated egg!" (p "This is an egg for CHICKEN 4, the unsupported old release.  You're almost certainly looking for " (int-link "/eggref/5/cjson" "the CHICKEN 5 version of this egg") ", if it exists.") (p "If it does not exist, there may be equivalent functionality provided by another egg; have a look at the " (link "https://wiki.call-cc.org/chicken-projects/egg-index-5.html" "egg index") ". Otherwise, please consider porting this egg to the current version of CHICKEN.")) (section 2 "chicken-cjson" (toc) (p (link "http://call-cc.org/" "Chicken Scheme") " bindings for the JSON parser " (link "https://github.com/DaveGamble/cJSON" "cjson") ". It cannot read from ports and must have the entire JSON object in memory. It also does not serialize to ports as " (tt "cjson->string") " returns a string.") (p "The " (tt "string->json") " procedure returns the same datastrucures as " (tt "medea") "'s, " (tt "read-json") ", so if you're passing that strings already, " (tt "string->json") " should be a drop-in replacement:") (highlight bash "    \n$ printf '{\"array\":[1,2,3],\"null\":null}' | csi -R cjson -p '(string->json (read-line))'\n((null . null) (array . #(1.0 2.0 3.0)))\n$ printf '{\"array\":[1,2,3],\"null\":null}' | csi -R medea -p '(read-json (read-line))'\n((array . #(1 2 3)) (null . null))") (p (tt "chicken-cjson") " may offer a significant performance improvement over " (link "http://wiki.call-cc.org/eggref/4/medea" "medea") " and " (link "http://wiki.call-cc.org/eggref/4/json" "json") ", but comes at a price: all data must be availabe as a string. This means you cannot parse JSON coming in from a port directly, and that's why there is no " (tt "read-json") " here.") (p (tt "chicken-cjson") " offers an alternative API which exposes a JSON c-struct as a " (tt "#<cjson>") " scheme record, and accompanying procedures like " (tt "cjson-string") ". This is for performance reasons and allows you to pick apart JSON objects using lolevel C-functions, without transitioning into the Scheme data-structure. This may be faster but is a lot uglier:") (highlight bash "    \n$ printf '{\"array\":[1,2,3],\"null\":null}' |\\\n  csi -R cjson -p '(cjson-double (cjson-array-ref (cjson-obj-ref (string->cjson (read-line)) \"array\") 1))'\n2.0") (p "Note that if you know that a number is fixnum, you can use " (tt "cjson-int") " instead.") (section 3 "Requirements" (p "None. " (link "https://github.com/DaveGamble/cJSON" "cjson") " comes bundled.")) (section 3 "TODO" (ul (li "Make " (tt "cjson-schemify") " faster yet by avoiding the O(n) procedures " (tt "cjson-obj-ref") " and " (tt "cjson-array-ref")) (li "Update to " (link "https://github.com/DaveGamble/cJSON" "cjson") " version " (link "https://github.com/DaveGamble/cJSON/releases/tag/v1.2.1" "1.2.1")) (li "Add support for modifying " (tt "cjson") " records like " (tt "cJSON_AddItemToArray")) (li "Make " (tt "with-cjson") " that allocates and deallocates in block, and does it fast?") (li "Make a common JSON API for all json parsers in CHICKEN?"))) (section 3 "API" (def (sig (procedure " (string->cjson string)" (id string->cjson))) (p "Note that that's not " (tt "string->json") "! Parses the string and returns a " (tt "#<cjson>") " record which holds a c-struct representing the JSON. The returned record has a finalizer attached to it so the underlying c-struct gets freed on garbage collection.")) (def (sig (procedure " (string->cjson* string)" (id string->cjson*))) (p "Like " (tt "string->cjson") " but does not attach a finalizer to the " (tt "#<cjson>") " record. " (tt "cjson-free") " must be explicitly called later on the returned value to avoid memory leaks. This is sometimes faster than attaching finalizers, particularly if there are large numbers of " (tt "#<cjson>") " objects.")) (def (sig (procedure " (cjson->string cjson [pretty-print?])" (id cjson->string))) (p "Convert the " (tt "#<cjson>") " object to its JSON-representation, returned as a string. " (tt "pretty-print?") " defaults to true. Note that this can only serialize " (link "https://github.com/DaveGamble/cJSON" "cjson") " records, and not scheme objects.")) (def (sig (procedure " (cjson-schemify cjson)" (id cjson-schemify))) (p "Convert the " (tt "#<cjson>") " object to scheme data-structures. The data-structures are the same as " (link "http://wiki.call-cc.org/eggref/4/medea" "medea") "'s, where " (tt "array => vector") " and " (tt "object => alist") ".")) (def (sig (procedure " (string->json string)" (id string->json))) (p "Make scheme data-structures of the json data in " (tt "string") " using [cjson]:") (highlight scheme "    \n(define (string->json str)\n  (let* ((j (string->cjson* str))\n         (s (cjson-schemify j)))\n    (cjson-free j)\n    s))") (p "For string inputs, this should be API-equivalent of " (link "http://wiki.call-cc.org/eggref/4/medea" "medea") "'s " (tt "(read-json)") ".")) (def (sig (procedure " (cjson-type cjson)" (id cjson-type))) (p "Pick out the type of a " (link "https://github.com/DaveGamble/cJSON" "cjson") " record. Returns a fixnum.") (pre "   [variable] cjson/false\n   [variable] cjson/true\n   [variable] cjson/null\n   [variable] cjson/number\n   [variable] cjson/string\n   [variable] cjson/array\n   [variable] cjson/object") (p "Exposes the " (link "https://github.com/DaveGamble/cJSON" "cjson") "-type constants.")) (def (sig (procedure " (cjson-int cjson)" (id cjson-int)) (procedure " (cjson-double cjson)" (id cjson-double)) (procedure " (cjson-string cjson)" (id cjson-string)) (procedure " (cjson-key cjson)" (id cjson-key))) (p "\"Unbox\" the json value. " (tt "cjson-type") " must match like this:") (highlight scheme "    \n(select (cjson-type cjson)\n   ((cjson/false) #f)\n   ((cjson/true)  #t)\n   ((cjson/null) 'null)\n   ((cjson/number) (cjson-double cjson))\n   ((csjon/string) (cjson-string cjson))\n   (else (error \"probably a vector or object\")))\n\n[procedure] (cjson-array-size cjson)") (p "Return the number of elements in the array. If " (link "https://github.com/DaveGamble/cJSON" "cjson") "'s type is not an array, this is undefined bahaviour.")) (def (sig (procedure " (cjson-array-ref cjson index)" (id cjson-array-ref))) (p "Return the element of " (tt "cjson") " at position " (tt "index") ". " (tt "index") " must be a fixnum. Undefined behaviour if " (tt "cjson") " is not an array.")) (def (sig (procedure " (cjson-obj-ref cjson key)" (id cjson-obj-ref))) (p "Select field " (tt "key") " from " (tt "cjson") ". Key must be a string. Undefined behaviour if " (tt "cjson") " is not of type " (tt "cjosn/object") "."))) (section 3 "Performance" (p "The performance characteristics of JSON parsing is mysterious. It is recommended to use " (link "http://wiki.call-cc.org/eggref/4/medea" "medea") " or " (link "http://wiki.call-cc.org/eggref/4/json" "json") " in most usage-cases because they can parse directly off ports and they also serialize. For particular cases, however, there may be significant performace improvements in using " (link "https://github.com/DaveGamble/cJSON" "cjson") ".") (p "Sometimes the speedup is negligible:") (highlight bash "    \n$ (echo '[' ; for i in {0..1000} ; do echo '\"str\", 1, 2, 3, 4,' ; done ; echo ' 0]') > bigjson\n$ time csi -R medea -e '(pp (read-json))' < bigjson >/dev/null\n\nreal    0m0.152s\nuser    0m0.140s\nsys     0m0.013s\n$ time csi -R cjson -e '(pp (string->json (read-string)))' < bigjson >/dev/null\n\nreal    0m0.114s\nuser    0m0.103s\nsys     0m0.010s") (p "Here, having to doing a " (tt "read-string") " first isn't great. " (link "https://github.com/DaveGamble/cJSON" "cjson") " may shine when you already have the JSON data as a string:") (highlight bash "    \n$ for i in {0..100000} ; do echo '{\"field\" : {\"id\" : 1}}' ; done > jsonlines\n$ time csi -R medea -R ports -e '(port-for-each (lambda (line) (pp (alist-ref `field (read-json line)))) read-line)' < jsonlines  > /dev/null\n\nreal    0m3.997s\nuser    0m3.933s\nsys     0m0.067s\n$ time csi -R cjson -R ports -e '(port-for-each (lambda (line) (pp (alist-ref `field (string->json line)))) read-line)' < jsonlines  > /dev/null\n\nreal    0m1.099s\nuser    0m1.083s\nsys     0m0.017s") (p "That's four times faster. In some cases, though, keeping the " (tt "#<cjson>") " record and using its combersome API can pay off:") (highlight bash "    \n$ JSON='{\"field\" : {\"id\" : \"ID\"} , \"a\":1, \"b\":2, \"c\":[1,{\"x\":{\"y\":\"y\"}},3],\"d\":{\"e\":[]}}'\n$ for i in {0..100000} ; do echo $JSON ; done > jsonlines\n$ for f in test*.scm ; do echo ===== $f === ; cat $f ; done\n===== test-cjson.scm ===\n(import cjson ports)\n\n(port-for-each\n (lambda (line)\n   (let ((cjson (string->cjson* line)))\n     (print (cjson-string (cjson-obj-ref (cjson-obj-ref cjson \"field\") \"id\")))\n     (cjson-free cjson)))\n read-line)\n\n===== test-medea.scm ===\n(import medea ports)\n\n(port-for-each\n (lambda (line)\n   (print (alist-ref 'id (alist-ref 'field (read-json line)))))\n   read-line)\n\n$ for f in test*.scm ; do echo $f ; time csi -s $f < jsonlines >/dev/null ; done\ntest-cjson.scm\n\nreal    0m0.541s\nuser    0m0.523s\nsys     0m0.020s\ntest-medea.scm\n\nreal    0m13.870s\nuser    0m13.717s\nsys     0m0.153s\n$ time jq '.field.id' < jsonlines  > /dev/null\n\nreal    0m0.513s\nuser    0m0.500s\nsys     0m0.017s") (p "In this particular run, " (tt "chicken-cjson") " is 25 times faster than medea, and performas about as well as " (link "https://stedolan.github.io/jq/" "jq") ". This speedup typically only happens where you are parsing a lot of JSON, but only a small part of that needs to go back into scheme. Also, this only works because we have one JSON object per line, effectively giving us a json-object delimiter."))))