decode error?

[muyinliu]

(json:decode-json-from-string "{"foo": [1, 2, 3], "bar":true}")
;((:FOO 1 2 3) (:BAR . T))

(json:encode-json-to-string (json:decode-json-from-string "{"foo": [1, 2, 3], "bar": true}"))
;"{"foo":[1,2,3],"bar":true}"

(json:decode-json-from-string "{"foo": [1, 2, 3]}")
;((:FOO 1 2 3))

(json:encode-json-to-string (json:decode-json-from-string "{"foo": [1, 2, 3]}"))
;"[["foo",1,2,3]]"

Last encode seems error. Why?

[agrostis]

The default encoder tries to encode Lisp lists as JSON arrays. In your second example, this is possible, because the inner list is encodable as a four-element array. In your first example, on the other hand, (:BAR . T) is not encodable (JSON has no evident means to represent a standalone cons cell), so the lists-as-arrays approach fails. Then the encoder tries a backup approach: check if the outer list consists exclusively of cons cells, and if so, encode it as a JSON object.

(The relevant piece of code is in ENCODE-JSON-LIST-GUESSING-ENCODER at https://github.com/hankhero/cl-json/blob/master/src/encoder.lisp#L197 .)

Generally speaking, the Lisp list being a polysemous data structure, we had to choose between the above strategy and its inverse: try encoding lists as objects, fall back to arrays. Both ways, you get what to a “naïve user” might appear as weird behaviour, but overall, the chosen strategy would produce less weirdness (or so it was felt intuitively).

[martinflack]

I'm running into a very similar problem.

JSON-USER> (decode-json-from-string "{\"k1\": {\"k2\": [\"val\"]}}")
((:K-1 (:K-2 "val")))
JSON-USER> (encode-json-to-string *)
"[[\"k1\",[\"k2\",\"val\"]]]"

I'm reading your rationale in the paragraph above. It's understandable to want to make decoding intuitive, but round-trip capability through decode+encode is a really desirable property for users as well - there are certain classes of tasks that require that property.

It's probably a case of tweaking the heuristics or adding more information to the decoded structure.

Decoding strings that are keys in JSON Objects as KEYWORD symbols in Lisp is quite elegant. Would it help to assume on encode that a KEYWORD should also be a key in a JSON Object, and try to favor that interpretation? Does that fix this?

[agrostis]

Well, that JSON keys are translated to Lisp keywords is actually a matter of customization, see variable JSON-SYMBOLS-PACKAGE. So, your approach should be reformulated as “encode alists whose members' CARs are symbols in JSON-SYMBOLS-PACKAGE to JSON objects”. It makes sense, although I'm somewhat concerned about performance.

[krwq]

I'm also hitting this:

(let ((json "{\"test\":[{\"test2\":{\"foo\":\"bar\"}}]}"))
         (format t "~a~%" json)
         (format t "~a~%" (cl-json:encode-json-to-string (cl-json:decode-json-from-string json))))

[martinflack]

I found some success by using the explicit mode. There is an explicit encoder macro included in CL-JSON which accepts keywords in the data to introduce certain Javascript entities. The following snippet will create an analogous decoder macro, although it may not be complete or the best way. Then you can round-trip the data because the ambiguity is removed.

(Perhaps CL-JSON could offer an official WITH-EXPLICIT-DECODER form?)

(in-package :cl-json)

(defmacro with-explicit-decoder (&body body)
  `(let ((end-of-array-handler  *end-of-array-handler*)
         (end-of-object-handler *end-of-object-handler*))
     (bind-custom-vars
       (:end-of-array  (lambda () (cons :array (funcall end-of-array-handler)))
        :end-of-object (lambda () (cons :alist (funcall end-of-object-handler))))
       ,@body)))

(let ((json "{\"test\":[{\"test2\":{\"foo\":\"bar\"}}]}"))
         (format t "~a~%" json)
         (format t "~a~%" (with-explicit-encoder (encode-json-to-string (with-explicit-decoder (decode-json-from-string json))))))