Thanks for the review @jmeas, and the encouraging comments. Typos fixed!

Thanks for rebasing your work and reopening this discussion, Ethan.

I'd like to try to establish consensus on some fundamentals before engaging in a thorough detailed review.

You have defined profile extensions as follows:

An API MAY support one or more "profile extensions" to supplement the capabilities of this specification.

A profile extension is a small, separate specification that defines a set of values that can be added to a JSON API document, and the meaning of those values at the different places they can occur.

My preference would be to define profile extensions as separate specifications that define particular usage patterns that are wholly compatible with the base JSON API specification.

I believe that profile extensions can be much more useful if they are allowed to define usage expectations of any specific part of the spec including query params, attributes, relationships, meta, etc. The only requirement is that these extensions must fully conform with the base spec.

Here are a few of the use cases we have discussed in the past:

• particular pagination patterns through specifying usage of the page param

• particular filtering patterns through specifying usage of the filter param

• inclusion of set data in top-level meta (e.g. pagination statistics in a partial set)

• inclusion of timestamps as either attributes or meta info per resource

• expectations about the nature of IDs - i.e. client or server generated, UUIDs, etc.

I am concerned that this PR does not allow for some of these use cases. Do you agree that they should be in scope? Do we have a fundamental disagreement on the purpose of profile extensions?

Hey Dan!

Good to be chatting with you again. And I'm glad we're backing up to look at the fundamentals before we get lost in the weeds.

I want to focus on 3 of your use cases, which I agree aren't neatly addressed by the current design:

• particular pagination patterns through specifying usage of the page param

• particular filtering patterns through specifying usage of the filter param

• expectations about the nature of IDs - i.e. client or server generated, UUIDs, etc.

I see the value in an API being able to communicate all of these things to clients. In my head, though, this has been a slightly different feature than profile extensions. In particular, most of these usage patterns apply on an API-wide level, so my assumption is/has been that they should ultimately be described in some sort of "home" document living at the root of the API. That document might reference some separate specifications, but these would be a different a type of extension than profile extensions.

A key distinction, I think, is whether the extension adds meaning to contents in the request/response document vs. whether it specifies things that are irrelevant to interpreting the current document, but might be useful in other ways (like in helping clients constructing future requests). To me, profile extensions are only about the former, whereas the extensions you're describing would also do the latter.

I guess we could make one type of extension that does both, but there are some ways in which, to me, they really do feel like different concerns. For example, a client might want to request that the server apply a profile extension (in its current definition) to a document, in order to get extra info, but it really doesn't make sense to have a client requesting/negotiating based on extensions describing API usage patterns.

About your other use cases...

inclusion of timestamps as either attributes or meta info per resource

I agree that extension being able to give meaning to an attribute would be nice. For example, a resource object like:

{ 
  "type": "examples",
  "id": "1",
  "attributes": { 
    // let's imagine both of these attributes have extension-defined meanings.
    // maybe the color extension defines a standard set of colors.
    "last-modified": "2017-07-24T20:08:45.497Z",
    "color": "green"
  }
}

is obviously nicer JSON than

{ 
  "type": "examples",
  "id": "1",
  "attributes": {},
  "meta": {
    "last-modified": "2017-07-24T20:08:45.497Z",
    "color": "green"
  }
}

and being able to keep the data in the attributes object is a plus. So, I'm all for changing the current design to support assigning meaning to attributes. And I think doing that shouldn't be too hard (it seems to amount to, in one way or another, letting the aliases functionality reach into the attributes bag).

One thing I want to preserve, though, is the ability for someone reading a JSON:API document to know which extensions are adding meaning to which document members, just based on the contents of the document. In the current design, if I see { "meta": { "last-modified": "..." } }, I know that the meaning of the value at the "last-modified" key is defined according to whatever extension is associated with the "last-modified" alias. I can then easily look up its meaning in that extension's spec. By contrast, if extensions aren't explicitly associated with the data they describe, I have to be familiar with all the extensions in use on the document to know which of them, if any, are defining a "last-modified" attribute.

Also, if each attribute isn't explicitly associated with an extension, it's possible for two extensions to give the same attribute contradictory meanings—either when the extensions are first created or, importantly, as they're updated over time to add new features. Part of the design goals I set for profile extensions was to allow extensions to evolve independently and without versioning schemes, and this isn't possible if multiple extensions can be applying to the same piece of data.

define usage expectations of ... relationships

As far as extensions applying to relationships: I think this is almost certainly just asking for a way to map relationship names to what RFC5889 would call "extended relation names", which I totally agree we should have (and which we've discussed a bit going all the way back to #159), but I think that's also a separate feature from profile extensions.

@ethanresnick - Thanks as always for engaging so thoughtfully!

In order to establish the boundaries of the responsibilities of profile extensions, I would find it helpful to step back and try to carve up the responsibilities of the "neighbors" to profile extensions (i.e. other extensions, schemas, profiles, home documents, etc. that help to define and clarify an API). Otherwise, we risk introducing an unintentionally "gerrymandered" concept prematurely (sorry for the tortured analogy).

To be clear, I don't want to take away from your hard work here and all the discussions that have taken place. I believe much of this work is useful even if its boundaries shift a little bit after discussion.

I would find it useful to start a new discussion (very soon!) with a goal of providing a 10K foot view of the potential ways to define and standardize API structure and capabilities (i.e. profile extensions and all its neighbors). If we can agree on the high order bits, we can empower the community to explore some of these areas, and we can come back to review your PR here with confidence.

In order to establish the boundaries of the responsibilities of profile extensions, I would find it helpful to step back and try to carve up the responsibilities of the "neighbors" to profile extensions (i.e. other extensions, schemas, profiles, home documents, etc. that help to define and clarify an API). Otherwise, we risk introducing an unintentionally "gerrymandered" concept prematurely (sorry for the tortured analogy).

Hey Dan! I also don't want us to ship something that's narrower than it needs to be and/or ultimately makes JSON:API's design unnecessarily fragmented. To avoid that, I'm open to having the high-level discussion you're describing, and will try to go into it not totally attached to the existing design.

That said, I think there's also huge value in just shipping something soon, as long as it gets us, say, 80% of the way there. Right now, there are a number of use cases that JSON:API isn't addressing, that us editors haven't had the bandwidth to address, and we're starting to see non-compliant implementations and (I'd imagine) people just abandoning JSON:API as a result. While the current profile extension design is intentionally limited, I do believe it would immediately unblock the community to start addressing many of these issues in ways that we can learn from, without being bound to adopt their solutions in the base spec. I don't want the perfect to be the enemy of the good here.

So, in the name of making sure we ship something, do you have ideas for how we can constrain this 10k foot conversation so it doesn't totally derail shipping something? I'd really like to figure out the parameters for such a conversation before we open what could be a giant can of worms.

@ethanresnick I agree that we've got to build some momentum. I'm willing to say that we should at the very least time box such a potentially sprawling open-ended conversation. I think we could hash some rough parameters out in a couple weeks. I'm willing to get the discussion started this week and try to build consensus by August 11.

I think it's critical that we proceed cautiously and deliberately, but that we do in fact proceed.

If there are other ways you'd like to constrain this conversation before we start, please let me know.

@dgeb Cool. Timeboxing this to August 11th sounds like a good solution to me. I think we should also define the scope and desired output a bit more, but I'll let you do that in your OP, since I want to make sure we're including all the things you think are important to include.

@ethanresnick Please see #1207 for my attempt at a high-level discussion about extending and describing the spec. Sorry that it took me longer than expected.

Just wanted to chime back in and say that I've since read your further comments in #1207 (comment) and am less convinced link templates could not handle our filters. Good explanations!

However, I'm still ever so slightly on the side of @dgeb that I would like profiles to have greater power to define the usage of reserved query parameters without adopting link templates.