Consider explaining projections in more detail

[springcomp]

Projections are one of the difficult concepts to grasp for newcomers and it can bite experienced users as well.
It needs better explanation in the documentation.

Also, the spec does not list all the projections. It should clearly indicate that projections are:

• list-wildcard-expression i.e *
• hash-wildcard-expression i.e [*]
• flatten-expression i.e []
• slice-expression i.e [0:1:1]

In contrast, an index ie [0] is not a projection.

[springcomp]

I think I have a mental model that works for explaining what are projections and how they work.

Imagine a JMESPath expression as a pipeline, like a series of more focused expressions that process input and produce output.
While the input and output of the whole JMESPath expression is always a JSON token, what flows through the pipeline can be thought of as either of two things:

• A JSON token
• A "projection"

I define a "projection" as being a list of JSON tokens (very similar to a JSON array).
But the difference is, while a JSON array is passed as a single value to the downstream expression in the pipeline, a "projection" sees its elements branch off individually to the downstream expression.

The only way to fold a projection back to a regular JSON array is to either use a | or to exit the pipeline. That is, if the result of the evaluation ends with a "projection", the JMESPath evaluation engine folds the result to JSON array.

So if you use a hash-wildcard-expression on an input JSON array, the result will be a projection, that is, a list of JSON elements, each of which will go through the downstream expression.

Consider the following input JSON document:

[
  { "name": "jack" },
  { "name": "john" }
]

Consider the following two very similar expressions:

• search( [*].name.length(@), …) -> [4, 4]
• search( [*].name|length(@), …) -> 2  

1. The input of both entire expressions are a JSON array.
2. After evaluating the first [*] expression in the pipeline, the output is a projection, i.e a list containing two JSON objects that must be projected to the downstream expression name.
3. Evaluating the identifier name happens twice, once for each element in the projection, and returns a new projection, i.e a list of two JSON strings.
4. Now, that’s where the two expressions differ.

Using . in a sub-expression

• When using a . in sub-expression, the two JSON strings in the projection are projected onto the length(@) expression.
• Each JSON string is sent as the first argument to the length() function which outputs a JSON number.
• Both JSON numbers are aggregated, the result being yet another projection, ie a list of two JSON numbers.
• Since this is the end of the entire expression, the JMESPath evaluation engine folds the projection to a regular JSON array.

Using | in a pipe-expression

• When using a | in a pipe-expression, however, the projection is folded back to a regular JSON array immediately.
• The JSON array contains two JSON strings, each of which is the value for the name properties in the original JSON document.
• This array is passed as the first argument to the length() function, which returns a JSON number.
• Since this is the end of the entire expression, that is what the JMESPath evaluation engine returns.

[springcomp]

Just realized this has already been done extremely well in our JMESPath Community version.

[springcomp]

jmespath/jmespath.site#108