Relative paths support

[bettio]

Proposal: JSONPath should have support to relative paths (to a current context), so it can be used as a building block for more advanced usages.

XPath makes a clear distinction between relative and absolute paths, e.g.:

given the following XML input:

<root>
    <e>foo</e>
    <a>
        <b>
            <c><d>1</d><e>bar1</e></c>
            <c><d>2</d><e>bar2</e></c>
            <c><d>3</d><e>bar3</e></c>
        </b>
    </a>
</root>

and the following template, which uses the relative XPath d and the absolute XPath /root/e:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:template match="/">
        <x>
            <xsl:for-each select="root/a/b/c">
                <item>
                    <y><xsl:value-of select="d"/></y>
                    <z><xsl:value-of select="/root/e"/></z>
                </item>
            </xsl:for-each>
        </x>
    </xsl:template>
</xsl:stylesheet>

the following XML can be produced:

<?xml version="1.0" encoding="UTF-8"?>
<x>
    <item>
        <y>1</y>
        <z>foo</z>
    </item>
    <item>
        <y>2</y>
        <z>foo</z>
    </item>
    <item>
        <y>3</y>
        <z>foo</z>
    </item>
</x>

I think that in a similar fashion JSONPath should allow relative paths as a building feature for more advanced usages.

Remark: I acknowledge that templating/transforms are out of scope here, however I believe that we must consider enabling more advanced usages.

Suggested implementation: JSONPath spec should mention that $ is always the document root, while any JSONPath which doesn't start with '$' is relative to the current context.
This wouldn't change the behavior for several existing implementations since they can assume that current context == document root.
Additional note: There is also a number of implementations for which a JSONPath without starting $ always evaluates to [], given the previous definition we can state that their current context is always set to null, therefore this change would allow both as valid JSONPath spec implementations.

[danielaparker]

Proposal: JSONPath should have support to relative paths (to a current context), so it can be used as a building block for
more advanced usages.

I think it already does, with paths starting with @, where @ represents the current node.

Suggested implementation: JSONPath spec should mention that $ is always the document root, while any JSONPath
which doesn't start with '$' is relative to the current context.

But consider

[{"true" : true}, {"false" : false}]

The expression, $[?(@.true == true)], which has a filter that evaluates a JSONPath expression relative to the current node @, and compares it to the JSON literal value true, gives

[{"true" : true}}]

If we allowed the current node symbol @ to be omitted, the expression would be ambiguous.

For comparison, JMESPath also uses the @ symbol to denote the current node, but requires that it be omitted except as a bare expression (by itself), otherwise understood. The comparable expression in JMESPath would be

[?true == `true`]

but JMESPath delimits JSON literals with "`", so the ambiguity does not arise.

Additional note: There is also a number of implementations for which a JSONPath without starting $ always evaluates to
[]

There are some implementations that allow the dot notation without root, e.g. ".foo", and more that allow the dot notation without root and dot, e.g. "foo". But in most cases, they are evaluating against the passed JSON value, and produce that same result as when starting with $.

Personally, I'm not enthusiastic about a JSONPath starting without a $ or @. My own view is that path syntax should follow common conventions, and should not vary depending on whether they appear in the main path or in expressions.

[bettio]

I think it already does, with paths starting with @, where @ represents the current node.

Actually it doesn't: the draft spec on this repo doesn't allow it at all, @goessner's spec doesn't mention it and the comparison doesn't test it.

So right now it is not obvious and a decision should be taken.

The expression, $[?(@.true == true)], which has a filter that evaluates a JSONPath expression relative to the current node @, and compares it to the JSON literal value true, gives

[{"true" : true}}]

If we allowed the current node symbol @ to be omitted, the expression would be ambiguous.

I agree this might be a problem, but it can be fixed by always requiring @ in filter expressions.
My point was mostly about relative paths in the outer expression (such as relative.foo.bar, where I didn't see yet an expression starting with @), not the filter expression.

I don't know if make any sense to distinguish between filter expression context (@) and the outer context, I think XPath doesn't so, we can be in feature parity quite easily from that point of view.

I remark that the outcome that I would expect from this discussion is reaching consensus about whether we want to support relative paths (to the current context) at first, then I would be glad to discuss how to support it (e.g. writing @.relative.foo.bar or relative.foo.bar or even .relative.foo.bar).

[danielaparker]

My point was mostly about relative paths in the outer expression (such as relative.foo.bar, where I didn't see yet an expression starting with @), not the filter expression.

I don't know if make any sense to distinguish between filter expression context (@) and the outer context, I think XPath doesn't so, we can be in feature parity quite easily from that point of view.

I still don't understand the requirement. The "outer context" is the root value at which JSONPath evaluation starts. Relative paths only have meaning inside the context of JSONPath evaluation. Within a filter expression, the current node changes as evaluation iterates over its parent. Relative paths starting with "@" can be evaluated against the current node.

In your original example, it seemed to me that you had a JSONPath expression, and in one case, you wanted to evaluate it against one value, and in another case, you wanted to evaluate it against another. In both cases you're going to evaluate a JSONPath expression against a value that you provide.

[gregsdennis]

I think we need to shelve this concept until after we have a first version out. As @bettio mentions, this is an advanced usage, but we can't even start to consider such things until we've defined basic usage.

[bettio]

I still don't understand the requirement. The "outer context" is the root value at which JSONPath evaluation starts.

JSONPath at the moment doesn't provide a mechanism for allowing users to choose whether their expression is relative to the JSON document root (absolute path) or to a any node inside that document.

This feature is crucial when JSONPath will be used inside of other software solutions / future specifications.

Let's think about how XPath is used inside of XSLT again:

<xsl:for-each select="root/a/b/c">
  <item>
    <y><xsl:value-of select="d"/></y>
    <z><xsl:value-of select="/root/e"/></z>
  </item>
</xsl:for-each>

xsl:for-each iterates over a collection of c tags, at each iteration the XPath current context is set to the xsl:for-each current item, however the document root is still accessible by using / prefix.

<z><xsl:value-of select="/root/e"/></z>
<y><xsl:value-of select="d"/></y>

here /root/e is always the same element because the / prefix tells the XPath evaluator that the path is relative to the document root, on the other hand the path d has no / prefix, therefore it is relative to the current context which is changed by the XSTL engine at each xsl:for-each iteration.

JSONPath right now doesn't allow this kind of flexibility using JSONPath language constructs (but it is required to workaround this limitation and it feels quite unnatural), I would expect absolute and relative paths concepts to be part of the specification, in a similar way to XPath.

this is an advanced usage

I don't agree here, when using XPath it is really a basic feature.

Quoting XPath doc:

There are two kinds of location path: relative location paths and absolute location paths.

A relative location path consists of a sequence of one or more location steps separated by /. The steps in a relative location path are composed together from left to right. Each step in turn selects a set of nodes relative to a context node. An initial sequence of steps is composed together with a following step as follows. The initial sequence of steps selects a set of nodes relative to a context node. Each node in that set is used as a context node for the following step. The sets of nodes identified by that step are unioned together. The set of nodes identified by the composition of the steps is this union. For example, child::div/child::para selects the para element children of the div element children of the context node, or, in other words, the para element grandchildren that have div parents.

An absolute location path consists of / optionally followed by a relative location path. A / by itself selects the root node of the document containing the context node. If it is followed by a relative location path, then the location path selects the set of nodes that would be selected by the relative location path relative to the root node of the document containing the context node.that would be selected by the relative location path relative to the root node of the document containing the context node.

Concerning my opinion I think that it would be a quite annoying limitation not having support for relative paths. I'm ok about postponing discussion on implementation details (such as which syntax would be better) to a future discussion, but I think that discussing at least if we want this feature or not is quite important.

So honestly I would expect reaching a consensus about whether we want somehow add this kind of feature to the spec, or leave it completely out as soon as possible.

[gregsdennis]

Okay I think I'm starting to understand what you want, @bettio. But please know that I'm coming at this with no knowledge on XPath except what you've described, so please bear with me.

Based on your example, XPath appears to be hierarchical in its declaration. Each level declares its own set of selectors. Because of this, yes, it's important that XPath support relative (acts on the currently-processed location) and absolute (acts on the root of the initial data argument) selection patterns.

JSON Path distinguishes between these using the @ (current location) and $ (root) operators.

It also appears that it's building a new XML object in the form in line with the structure of the XPath query. This is describing transformation, which is where JSON Path and XPath deviate.

JSON Path does not build a new JSON value. It is only intended for finding data, not constructing new values from that data. XPath is doing both of these things at the same time.

I think this is where the "XPath for JSON" slogan breaks down, as has been discussed in #65.

[bettio]

Okay I think I'm starting to understand what you want, @bettio. But please know that I'm coming at this with no knowledge on XPath except what you've described, so please bear with me.

Sorry, my fault, I'll try to be clearer here.

I'll introduce XPath and XSLT and their relationship here (by quoting w3c):

XSLT, which is a language for transforming XML documents into other XML documents.

A transformation in the XSLT language is expressed as a well-formed XML document [XML]

XSLT makes use of the expression language defined by [XPath] for selecting elements for processing

Going back to my example: "/", "root/a/b/c", "d" and "/root/e" are the XPath espressions embedded into the XSLT document.

So my point is about enabling embedding of JSONPath expressions into different technologies.

JSON Path does not build a new JSON value. It is only intended for finding data, not constructing new values from that data.

Basically I'm not asking/meaning this.

My point is about enabling users writing both paths relative to a context (which is defined outside JSONPath) and absolute paths (which refer to the document root by using $).

At the moment I'm not strongly opinionated about how we are going to do this, but let's suppose just for a moment we are going enabling it by using `@'.

Therefore given:

{
  "d": "bar",
  "e": "foo",
  "arr": [{"d": 1, "e": "test1"}, {"d": 2, "e": "test2"}, {"d": 2, "e": "test2"}]
}

If we choose $.arr[0] as context (which is {"d": 1, "e": "test1"}) I would expect following JSONPaths evaluating to:

• $.e -> ["foo"]
• $.arr[0].e -> ["test1"]
• @.e -> ["test1"]
• $.d -> "bar"
• @.d -> [1]

Remark: I don't care if we are going to use @ or € for this purpose, I just care about allowing writing two different kind of paths (as XPaths does) that allow writing expressions equivalent to the ones I wrote before.

[gregsdennis]

Thank you for the detailed summary and explanation.

Looking at your example of what you expect to happen at the end of your comment, this is precisely what the current syntax does within the context of evaluation.

However, @ is not valid at the beginning of a standalone path. It seems this is what you're asking for.

As an aside, I think there's a further deviation between proper JSON (spec) and XML in that JSON doesn't have the idea of a top-level document. Because of this, tooling can just pass whatever current JSON value it has to a proper path evaluator to get the required output. This is the current state of things, which is what we're actively trying to define at this time.

However, I can see the benefit of allowing a path to start with @ to indicate to such tooling which JSON value. I believe some implementations support this as well (to be verified). Basically your XML example from earlier in a hypothetical JSONT could be

​{
  "for-each​": {
    ​"select": "​$.a.b.c​"​,
    "item​": {
      "y​": "@.​d​",
      "z​": "$.e​"​
    }
  }
}

The difference between @.d and $.e becomes immediately apparent and is more of a cue to the JSONT implementation (which knows the difference between the JSON value it was given and the current JSON value) than the JSON Path evaluator.

But this only works if the JSON Path evaluator allows starting a path with @. So that's your proposal.

---

Proposal: allow paths to start with @. It operates no differently than $, but can serve as an important indicator to tooling and other systems that consume JSON Path.

Moreover, I believe it would help define such paths when defined within filter expressions. (See #17 & #64.)

[danielaparker]

@gregsdennis wrote:

Okay I think I'm starting to understand what you want, @bettio.

Me too :-)

In JSON equivalent, you want to transform

[
    "foo",
    {"a" : {
        "b" : [
            [{"d" : 1}, {"e" : "bar1"}],
            [{"d" : 2}, {"e" : "bar2"}],
            [{"d" : 3}, {"e" : "bar3"}]
        ]
    }
    }
]

into

[
    {
        "y" : 1,
        "z" : "foo"
    },
    {
        "y" : 2,
        "z" : "foo"
    },
    {
        "y" : 3,
        "z" : "foo"
    }
]

As @gregsdennis mentioned, JSONPath doesn't support that and never will, JSONPath will only select items that already exist in the original JSON document, it will not create new items.

You might want to look at JMESPath instead. JMESPath is a query language for transforming JSON documents into other JSON documents. It's supported in both the AWS and Azure CLI and has libraries available in a number of languages.

Daniel

[bettio]

As @gregsdennis mentioned, JSONPath doesn't support that and never will, JSONPath will only select items that already exist in the original JSON document, it will not create new items.

I think you are missing my point, I think I stated that already a few times, but I'm going to remark it: I'm not looking for a solution for transforming nodes.

Please, take a look to the example (the example about expressions such as @.e) that I reported here, which is not about transforming.

Please, let me know if this reply was helpful, or if you need any further clarification on this topic.

[gregsdennis]

@danielaparker, @bettio isn't asking for JSON Path to provide transformations. He's asking for JSON Path to allow @ at the beginning of a path so that, separately, other tooling can use JSON Path. Transformations are just an example of such tooling that could take advantage of this.

It's opening a door for others, that's all.

Please see my proposal rephrase at the end of my comment.

[danielaparker]

@gregsdennis wrote:

Proposal: allow paths to start with @. It operates no differently that $, but can serve as an important indicator to tooling an other systems that consume JSON Path.

Okay, I agree with that, so we may now be up to a minority of two :-)

[timbray]

I don't think this is compatible with our charter?

[gregsdennis]

How so? It's not removing any functionality of existing implementations, and it actually helps define some things in expression evaluation.

[danielaparker]

@gregsdennis wrote:

How so? It's not removing any functionality of existing implementations, and it actually helps define some things in expression evaluation.

Just to note, while not documented as a feature, the influential Jayway implementation accepts paths that start with '@', most likely for the reason that @gregsdennis mentioned, to help with expression evaluation in filter predicates. You can check it out here.

[bettio]

Just to note, while not documented as a feature, the influential Jayway implementation accepts paths that start with '@

Jayway implementation is indeed influential, just some numbers for the records: here on GH they have 5.7k stars, 1.1k forks, 50 contributors and it used by 1236 artifacts on maven.
There is clearly a huge ecosystem that builds on it and we cannot ignore it.

There is another good point about starting with @: it would simplify grammar for my implementation (I would love to remove that special case TBH) and more importantly I think that would be beneficial for other implementations too.

IMHO that might simplify the specification as well because we would remove any special case for @.

[bettio]

So, which are the next steps to make this part of the draft?

[cabo]

Write up a PR?

Once that is in place and has had some discussion (and fixes), take the PR to the mailing list.
Ideally, this will result in a short discussion (because most discussion has been on the issue and the PR already), and either the editors pick it up right away or the chairs state consensus (possibly aided by an explicit consensus call).

[bettio]

Write up a PR?

Ok, tomorrow I'll start working on it.

Thanks for all your feedbacks, it has been helpful discussion.

[glyn]

I don't see a compelling need for this feature.

Furthermore, it seems to increase the surface area of the spec in a way which, as mentioned above, isn't compatible with our charter:

The WG will develop a standards-track JSONPath specification that
is technically sound and complete, based on the common semantics
and other aspects of existing implementations. Where there are
differences, the working group will analyze those differences and
make choices that rough consensus considers technically best, with
an aim toward minimizing disruption among the different JSONPath
implementations.

If only a small number of implementations, however influential, have a particular semantics, then standardising those semantics will fail to minimise disruption among the other implementations.

[danielaparker]

I don't see a compelling need for this feature.

I'm interested in this issue, partly because I'm on the yay side (with reasons), but more importantly I'm interested to observe how the WG resolves matters that have some participants on the yay side , some on the nay, and presumably others that don't care. As far as I can tell, for all the issues that have arisen that find participants on one side or the other, none have been resolved with a considered decision. Of course, it's hard to tell where consensus lies, since so few people take part in discussions.

Furthermore, it seems to increase the surface area of the spec in a way which, as mentioned above, isn't compatible with our charter:

The WG will develop a standards-track JSONPath specification that
is technically sound and complete, based on the common semantics
and other aspects of existing implementations. Where there are
differences, the working group will analyze those differences and
make choices that rough consensus considers technically best, with
an aim toward minimizing disruption among the different JSONPath
implementations.

If only a small number of implementations, however influential, have a particular semantics, then standardizing those semantics will fail to minimize disruption among the other implementations.

As a principle, I think that requires interpretation. Taken literally, it would make problematic some of what's in the draft, such as requiring that slices can appear as elements in union expressions, see Union with slice and number, less than half of implementations (17 out of 41) support that, in particular, the important Jayway implementation doesn't have that, so a vast part of the JSONPath ecosystem doesn't have that.

Realistically, legacy implementations are unlikely to introduce breaking changes regardless of what's in the draft. For example, the legacy Java ecosystem is not going to change the meaning of a filter applied to a JSON object, even though it's an outlier. But generalizations of the notation, such as allowing a path to begin with @, or allowing slices to appear as union elements, are not breaking changes. It is conceivable, although far from a certainty, that a legacy implementation could adopt some of these. But it's the new implementations, or existing ones with a comparatively small userbase, that we can expect to follow the specification. Perhaps in ten years we might find widespread adherence to the specification. In any case, I trust nobody believes that it is possible to have an interoperable specification that does not fail to comply with some feature of every existing JSONPath implementation. It's apparent already in the few lines of grammar in the draft, it will become even more apparent when the WG tackles filter predicates.

[cabo]

As far as I can tell, for all the issues that have arisen that find participants on one side or the other, none have been resolved with a considered decision. Of course, it's hard to tell where consensus lies, since so few people take part in discussions.

Most of the issues raised aren't ready for a decision. I do expect the chairs to pick up some issues and issue consensus calls (or declare consensus). But, really, deciding on nested queries or on the regexp flavor is way too early. We need to work on basics like the processing model first. But we can continue to collect data on issues like the present one.

[timbray]

What cabo said. There are high-quality arguments being offered. I would specifically recommend that everyone have a glance at our charter and keep that goal in the back of your mind.

…

On Sat., Mar. 20, 2021, 8:25 a.m. cabo, ***@***.***> wrote: As far as I can tell, for all the issues that have arisen that find participants on one side or the other, none have been resolved with a considered decision. Of course, it's hard to tell where consensus lies, since so few people take part in discussions. Most of the issues raised aren't ready for a decision. I do expect the chairs to pick up some issues and issue consensus calls (or declare consensus). But, really, deciding on nested queries or on the regexp flavor is way too early. We need to work on basics like the processing model first. But we can continue to collect data on issues like the present one. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#59 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEJEZM5BGJHAGWG7AH6H3TES45VANCNFSM4Y3YOOQA> .

[bettio]

I don't think it is just a Java Jayway implemention peculiarity, and I did a quick investigation, @.a returns a valid result ([1]) for {"a": 1} at least with following implementations [1] (which I took from cburgmer's list with the precious help of his tool).

@glyn

If only a small number of implementations, however influential, have a particular semantics

Which is the same number of implementations supporting regex, if they are not enough, I'm worried that the same argument/rule would stop us from adding support to regex. Honestly I think they are enough to discard this stopper.

@glyn

then standardising those semantics will fail to minimise disruption among the other implementations.

Standardising @.foo doesn't disrupt existing JSONPath expressions, this is not a breaking change.
Furthermore support for @.foo can be added by simplifying existing code (at least this hold true for ExJSONPath), that can be added by removing the special case for supporting @ only inside filter expressions.

@glyn

Furthermore, it seems to increase the surface area of the spec in a way which, as mentioned above, isn't compatible with our charter:

Actually my PR formalize what those [1] implementations already support by adding just few lines of text, and it can be sumarized with:

-json-path = root-selector *selector
+json-path = start-selector *selector
+start-selector = root-selector / current-item-selector
root-selector = "$" ; $ selects document root node
+current-item-selector = "@" ; @ selects current node

Moreover if we continue the comparison with the JSONPath regex standardization (=~), I belive that the spec-surface-area / number-of-implementations is much more worse on the regex side. So there is a lot of effort for supporting a feature that is implemeted by 11 implementations.

Honestly I don't belive that blindly applying struct rules is good for us.

The WG will develop a standards-track JSONPath specification that
is technically sound and complete, based on the common semantics
and other aspects of existing implementations. Where there are
differences, the working group will analyze those differences and
make choices that rough consensus considers technically best, with
an aim toward minimizing disruption among the different JSONPath
implementations.

I believe that we can agree that standardising @.foo is compatible with it, and there are no preconditions against standardising it for the reasons I explained and I'll further explain.

Furthermore:

• Supporting @.foo will make filters specification simpler, since we can just use json-path grammar definition instead of a custom one for filters.
• As @gregsdennis said, we need to introduce and define scopes for @ anyway (which are required for nested filters among all)
• This feature didn't came out of the blue, it has a simple XPath equivalent

Furthermore about XPath:
the spec:

"The JSONPath tool in question should" ... "cover only essential parts of XPath 1.0".

Relative paths (such as foo) are an essential part of XPath, and I think that XPath cannot be used as foundational technology for a number of specs that embeds it (such as XSLT).

I don't see a compelling need for this feature.

I'll quote @gregsdennis (he described it better than me):

He's asking for JSON Path to allow @ at the beginning of a path so that, separately, other tooling can use JSON Path. Transformations are just an example of such tooling that could take advantage of this.
It's opening a door for others, that's all.

@gregsdennis:

Proposal: allow paths to start with @. It operates no differently than $, but can serve as an important indicator to tooling and other systems that consume JSON Path.

I wish to use JSONPath as a foundational standard to build more advanced tools and specs in the future, rather than trying to find a "feature poor" minimum subset. If we fail doing so we should rather stick to JSON pointers.

@cabo I don't think it is too early to discuss this specific issue, furthermore it might be helpful when describing filters.
The change I proposed it is quite self contained in few lines of diff, and in my opinion doesn't involve complex decisions such as regex or the filtering language.

@glyn, @timbray please, let's move forward and discuss any potential issue with @.foo rather further discussing if this is perfectly compatible with a littleral interpration of our charter.

[1]:
Ruby (1/1 implementation):

• jsonpath

Perl (1/1 implementation):

• JSON-Path

Dart (1/1 implementation):

• json_path

Objective-C (1/1 implementation):

• SMJJSONPath

Java (1/2 implementations):

• com.jayway.jsonpath

dotNET (2/4 implementations):

• Manatee.Json
• JsonPath.Net

Go (3/6 implementations):

• github.com-oliveagle-jsonpath
• github.com-ohler55-ojg
• github.com-spyzhov-ajson

As Elixir implementation maintainer I'm willing to implement it aswell ;)

[glyn]

I'm interested in this issue, partly because I'm on the yay side (with reasons), but more importantly I'm interested to observe how the WG resolves matters that have some participants on the yay side , some on the nay, and presumably others that don't care. As far as I can tell, for all the issues that have arisen that find participants on one side or the other, none have been resolved with a considered decision. Of course, it's hard to tell where consensus lies, since so few people take part in discussions.

Me too. I haven't yet internalised https://tools.ietf.org/html/rfc7282.

[danielaparker]

Note: As it seems this issue is about to be closed, I've moved some content from this comment to #76.

Just to summarize my earlier comments on the meaning of "current node", and reasons why JSONPath expressions starting with "@" are justifiable.

First, consider the analogous notions of "current object", "current node", and "context item" in Goessner JSONPath, JMESPath, and XPath 3.1.

	Goessner JSONPath	JMESPath	XPath 3.1
What is it called?	"current object"1	"current node"2	"context item" (or "context node" if the context item is a node)3
Where is it defined?	Goessner	JMESPath Specification	XPath 3.1 W3C Recommendation
How is it represented?	@	@	.
How is it defined?	"using the symbol '@' for the current object"	"The current-node token can be used to represent the current node being evaluated...At the start of an expression, the value of the current node is the data being evaluated by the JMESPath expression. As an expression is evaluated, the value the the current node represents MUST change to reflect the node currently being evaluated. "	"The context item is the item currently being processed. When the context item is a node, it can also be referred to as the context node. The context item is returned by an expression consisting of a single dot (.)."
How does it appear in the grammar?		current-node = "@"	ContextItemExpr | ::= | "."
Explicit or understood?	Always explicit	Usually understood - "JMESPath assumes that all function arguments operate on the current node unless the argument is a literal or number token. Because of this, an expression such as @.bar would be equivalent to just bar, so the current node is only allowed as a bare expression."	Sometimes explicit - .//para selects the para element descendants of the context node. Sometimes understood - para selects all para children of the context node
How is it used?	$.store.book[(@.length-1)].title $.store.book[?(@.price < 10)].title	foo[].[count(@), bar]4	.//para /books/book[fn:count(./author)>1])

1. In Goessner JSONPath, the term "object" used here seems to refer to any JSON value. The term "node" is only used when referring to XPATH 1.
2. In JMESPath, the term "node" is undefined, but can be considered synonymous with "JSON value". In JMESPath, paths (locations) don't figure, and results may be elements in the original JSON document, or newly created elements that don't exist in the JSON document.
3. For XPath 3.1, "node" is defined in the XQuery and XPath Data Model 3.1 W3C Recommendation
4. In this JMESPath expression, foo is understood to mean @.foo, and bar is understood to mean @.bar. The only place where the symbol @ is used explicitly is as a "bare expression", as shown in the argument to the count function.

"current object" in Goessner JSONPath as represented by @ is clearly underspecified, but can easily be understood in the more general sense in JMESPath, as "the node currently being evaluated", or in XPath 3.1, as " the item currently being processed".

The Goessner examples:

$.store.book[(@.length-1)].title                   (1)

$.store.book[?(@.price < 10)].title               (2)

are fully consistent with the more general sense. In (1), the "current node" @ represents the array $.store.book, in (2), the ? connotes iteration over the array, and @ represents an array element.

It's easy to generalize that at $, the current node @ represents '$'; at $.store, it represents $.store, and so on.

[timbray]

Has someone done a survey of which implementations support leading @? I thought I saw one in the email/github trail but now I can't find it. Obviously relevant given our charter.

…

On Sun, Mar 21, 2021 at 9:39 AM Daniel Parker ***@***.***> wrote: Just to summarize my earlier comments on the meaning of "current node", and reasons why JSONPath expressions starting with "@" are justifiable. First, consider the analogous notions of "current object", "current node", and "context item" in Goessner JSONPath <https://goessner.net/articles/JsonPath/>, JMESPath <https://jmespath.org/specification.html#current-node>, and XPath 3.1 <https://www.w3.org/TR/xpath-31/#eval_context>. Goessner JSONPath JMESPath XPath 3.1 What is it called? "current object" "current node" "context item" (or "context node" if the context item is a node ) Where is it defined? Goessner <https://goessner.net/articles/JsonPath/> JMESPath Specification <https://jmespath.org/specification.html#current-node> XPath 3.1 Specification <https://www.w3.org/TR/xpath-31/#eval_context> How is it represented? @ @ . How is it defined? "using the symbol '@' for the current object" "The current-node token can be used to represent the current node being evaluated...At the start of an expression, the value of the current node is the data being evaluated by the JMESPath expression. As an expression is evaluated, the value the the current node represents MUST change to reflect the node currently being evaluated. " "The context item is the item currently being processed. When the context item is a node, it can also be referred to as the context node. The context item is returned by an expression consisting of a single dot (.)." How does it appear in the grammar? current-node = "@" ContextItemExpr | ::= | "." Explicit or understood? Always explicit Usually understood - "JMESPath assumes that all function arguments operate on the current node unless the argument is a literal or number token. Because of this, an expression such as @.bar would be equivalent to just bar, so the current node is only allowed as a bare expression." Sometimes explicit - .//para selects the para element descendants of the context node. Sometimes understood - para selects all para children of the context node How is it used? ***@***.***)].title ***@***.*** < 10)].title foo[].[count(@), bar] .//para "current object" in Goessner JSONPath as represented by @ is clearly underspecified, but can easily be understood in the more general sense in JMESPath, as "the node currently being evaluated", or in " the item currently being processed". The Goessner examples: ***@***.***)].title (1) ***@***.*** < 10)].title (2) are fully consistent with the more general sense. In (1), the "current node" @ represents the array $.store.book, in (2), the ? connotes iteration over the array, and @ represents the array element. It's easy to generalize that at $, the current node @ represents '$', at $.store, it represents $.store, and so on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEJE6K6YGIL545TLZ5OIDTEYOMPANCNFSM4Y3YOOQA> .

[danielaparker]

@timbray wrote:

Has someone done a survey of which implementations support leading @? I thought I saw one in the email/github trail but now I can't find it. Obviously relevant given our charter.

See @bettio's post above. He's determined that 10 of the 41 implementations in Christoph Burgmer's JSONPath Comparisons support leading @, including the important Jayway implementation.

I would suggest that he submit his test case as a pull request to JSONPath Comparisons, so that it can be properly referenced.

[goessner]

I think, the json_path_eval(root, root.foo, "@.bar") example makes the usefulness of relative pathes for tools cristal clear. So current-item-selector requires either

• an explicit absolute path as start node (here root.foo) or
• an implicite current node given during processing the JSON tree
• else the fallback to root-selector $.

The obvious problem we are having here, is above tooling being completely out of scope of the draft. At best we can demand, that a starting relative path always needs as compagnon an explicit absolute path (wherever it comes from) or an implicite one defaulting to $.

We then have by combining both an overall absolute start path again, as @bettio showed in his starting post as XPath analogon:

which uses the relative XPath d and the absolute XPath /root/e:

So once formulated with care – in work at #75 – it does not cost much.

But ...

... current item selector `@` when used outside a filter ...

needs to be discussed more urgently before. So I would like to wait with merging pull request #75 until all uses of @ are readily discussed.

[bettio]

@goessner the one you quoted is the older #75 wording, I updated it in the meantime to:

The current item selector `@` can be used to represent the current item being
evaluated. The initial current item defaults to the root node unless a
descendant node is given to the evaluator as initial current item.

I did that so we can merge it before going deep into the filters formalization that I think it will require a lot of work and time.

I agree with you that we need to discuss in depth the meaning of @ and investigate it more.

If we all agree I propose following action points:

1. agreeing that relative paths are a wanted feature that should be part of the standard
2. closing this issue (I feel like that this issue is to long to read, and we should start with a new issue/discussion)
3. starting a new issue for discussing @ and investigating it in depth

I don't know if @glyn , @cabo or @timbray have any further argument or point on the feature itself, otherwise if we have no further points we can move forward :)

@timbray I feel like that the charter point is not an issue anymore and we can all agree that it is compatible.

By the way, thank you to all of you for involvement in this discussion, it has been an interesting discussion.

[glyn]

I would say that filters are the main motivation for @ and we should keep this issue open, but defer further work on it until filters have been added to our draft. The interim position can be that @ is meaningless outside a filter - then we are free to extend to usages outside filters if we decide that's something we want to pursue.

[bettio]

@glyn sorry, I feel a bit like we are indefinitely deferring any decision rather than discussing actual arguments against this use case.
If you have any argument I wish to discuss it, if there aren't I think we can agree all and move forward.

Also as I commented before, I think that actually allowing @ outside filters enable us into writing a simpler grammar for filters, so I believe we should discuss @ usage before of them.

[glyn]

My reasons for not favouring the above use of @ outside filters are:

• Our charter. How many implementations allow a "current value" to be passed in rather than merely supporting "leading @"?
• This would increase the surface area of the spec (i.e. allowing a "current value" to be passed in) without a compelling use case. In other words, this feels like a solution looking for a problem. ;-)

Also, I think arguments about simplifying the grammar of filters are moot until we have such a grammar.

[bettio]

I'm "reopening" this discussion after the discussion we had in other issues, I think we are ready to further discuss this one.

@glyn:

Our charter. How many implementations allow a "current value" to be passed in rather than merely supporting "leading @"?

As I mentioned, I think we all clarified now, in other discussions, that this is not an issue and it is 100% compatible with our charter.

By the way ExJSONPath implements it ;) I also did a quick check (only) on Jayway Java and it has the following function:

public EvaluationContext evaluate(Object document, Object rootDocument, Configuration configuration, boolean forUpdate)

However I'm not a Java developer so I'm not sure how this is exposed through the API.

Honestly I don't plan to check all ther other implementations, since it is not required/useful for our discussion (again it is not a WG charter requirement and also it is a time consuming activity).

@glyn:

This would increase the surface area of the spec (i.e. allowing a "current value" to be passed in)

It is completely optional, implementations will not be required to follow this part of the spec and they can stick to the document root as default initial current value.
The surface increase is merely telling implementers that current item can be (if they want / if they need it) set to a descendant node as initial node (instead of the root node).
I also think that defining an API is out-of-scope, but mentioning that a different (than the document root) initial current item can be chosen is fine.

@glyn:

without a compelling use case.

I got following feedbacks:

@gregsdennis

However, I can see the benefit of allowing a path to start with @ to indicate to such tooling which JSON value. I believe some implementations support this as well (to be verified).

@gregsdennis:

It's opening a door for others, that's all.

@danielaparker:

Okay, I agree with that, so we may now be up to a minority of two :-)

@goessner:

I think, the json_path_eval(root, root.foo, "@.bar") example makes the usefulness of relative pathes for tools cristal clear.

It looks to me that there is some agreement about its usefulness.

@glyn:

In other words, this feels like a solution looking for a problem. ;-)

@gregsdennis made an example (similar to my usecase) that vastly benefit from a similar feature.

Also again this is about "opening a door" we don't need to know all the possibilities in advance.

As a "philosophical" point of view I also believe that the relative path concept is a widely known concept tight to the path concept. Nearly every time some kind of path is defined, a relative path concept is defined as a consequence as well.

Last but not least, this feature has a XPath 1.0 counterpart so we are not introducing a new feature out of the blue, and we already know that it is a widely used feature in XPath.

Also, I think arguments about simplifying the grammar of filters are moot until we have such a grammar.

We already know that it will (I know it because I had the same kind of issue when I was working to my implementation), json-path rule with my changes can be used as-is for a (likely) filter grammar definition, since it provides a uniform definition for both @.foo and $.bar cases, which doesn't require additional filter specific path definitions.

[glyn]

I still think the complexity/benefit ratio of this proposal is quite high, so I'm not keen to adopt it. Let's see what the consensus is.

[gregsdennis]

I still firmly support this.

[bettio]

No new activity in last month.
@glyn are you willing to compute consensus based on the whole discussion?

[glyn]

No new activity in last month.
@glyn are you willing to compute consensus based on the whole discussion?

No, I'll leave that to the Chairs, thanks.

[bettio]

Any news?

[ghost]

@bettio the consensus that was made during the meeting on Tuesday (minutes here) was that at this stage we won't include relative path support as it's been described in this PR. Several points already mentioned in this issue were re-iterated in the meeting such as charter scope, and percentage of existing deployments.

If you continue to feel strongly for it the two options are either providing a further case for its inclusion into the base specification, or consider this work as an extension - a key action that was set at the meeting was for JSONPath to have an extension model in place, and this seems like an obvious candidate.

[gregsdennis]

https://stackoverflow.com/q/68926463/878701

This is a question that would benefit from another concept of relative paths, more akin to the Relative JSON Pointer. This pointer allows travel up the data structure so that parent, sibling, and cousin data can be selected.

[danielaparker]

@gregsdennis, access to the parent node is one of the most requested features of JSONPath on Stack Overflow and elsewhere. JSONPath Plus was first to introduce the notation '^' for parent operator. Currently this feature is supported by three implementations in the JSONPath Comparisons, see Filter expression with parent axis operator. See Support for path axis navigation for discussion pertaining to this query, and why the implementations give different results for values. (They do give the same results for output paths, it appears that JSONPath Plus gives inconsistent results for returning output paths and values when applying the parent operator.)

Support for a JSONPath parent operator doesn't actually require keeping a parent pointer along with the JSON value. Rather, all it requires is path tracking (which is required anyway for implementations that support output paths), the ability to back up one level in the path, and access to the root node. In other words, it can be implemented trivially in all implementations that support output of normalized paths as an option.

[goessner]

IIRC we decided for filter expressions containing filter expressions ... can't locate that discussion though.

So with a former example of mine

Consider a school,

{
   "students": [...],
   "staff": [...],
   "teachers": [...]
}

while looking for the array(s) containing persons named 'Müller' ...

we can now write $[?@[[email protected]=='Müller']].

Obviously the corresponding filter syntax in the latest draft

https://ietf-wg-jsonpath.github.io/draft-ietf-jsonpath-base/draft-ietf-jsonpath-base.html#name-filter-selector

should be corrected from

rel-path = "@" *(dot-selector / index-selector)

to

rel-path = "@" *(dot-selector / index-selector / filter-selector)

then.

Given this, an explicit parent operator ^ would be redundant. But ...

@danielaparker

Support for a JSONPath parent operator doesn't actually require keeping a parent pointer along with the JSON value. Rather, all it requires is path tracking (which is required anyway for implementations that support output paths), the ability to back up one level in the path, and access to the root node. In other words, it can be implemented trivially in all implementations that support output of normalized paths as an option.

... regarding the obvious low hurdle for implementating that operator, we might discuss it under the aspect of user-friendliness.

[goessner]

sorry ... wrong button ...

[cabo]

See also 112 discussion on #109.
Closing now.

[gregsdennis]

I found another use case for this.

I'm building an extension vocabulary for JSON Schema in which users will be able to define a JSON Path (per the spec) to query instance data for use during validation.

For example:

{
  "$schema": "https://json-everything.net/meta/data-2023",
  "type": "object",
  "properties": {
    "options": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "integer" },
          "value": { "type": "string" }
        }
      }
    },
    "selection": {
      "data": {
        "enum": "$.options[*].id"   // this is my extension functionality
      }
    }
  }
}

This schema expects an instance (the data being validated) to have two properties: options and selection. options is an array of objects, each of which contains an id property, and selection needs to be one of those id values.

For this particular case, I don't have a problem, but if I were validating a value nested into the instance (JSON Schema does track its location within the instance already), I might want to be able to either query the current location or the entire instance. Having a relative path (i.e. starting a path with @) would allow the schema author to specify this.

The schema author could use @.options[*].id to search at the current location, or $.options[*].id to search from the root.

This just allows more flexibility.

[cabo]

Hi Greg,

indeed, co-occurrence constraints are an important use-case for XPath-like navigational constructs.
This need is well understood in the YANG community, which is stuck with XPath, which doesn't make it easy to actually arrive at the right place before selecting downwards again. So there are lots of buggy YANG specifications out there that get this wrong. (Exactly the same problem that made us stop trying to use relative JSON pointers in SDF.)

So yes, this is an important use-case, but I'm not sure it is one for "this" -- we need better solutions than "three blocks straight, then left, then four more blocks straight, then left straight at the fork, and if you don't see a music store on the right hand side after two more blocks, ask again".

[gregsdennis]

I'm not sure that relative paths have the same problems you're describing. The issue you're describing from Relative JSON Pointer (which I do have an implementations of), is the backtracking and side-stepping functionality. JSON Path offers neither of those, and that's not being requested.

What's being requested here is merely the ability to specify (with the path itself) whether the query should begin at the data root ($) or a location within the data (@). If the latter, then it becomes necessary that the party running the query would provide the starting location.