Specify/advise semantics of default Container resources (index.html/.ttl)

A topic that causes a lot of implementor and app-developer confusion is the handling of default resources for Containers - index.html and/or index.ttl.

Questions:

Is the index.html behavior a MUST or MAY?
Is the index.ttl behavior a MUST or MAY?
How to prevent the fairly common occurrence of "I accidentally created an index.html but forgot to add an .acl, plus that kicks me out of the Data browser, so anyways what do I do now."

In either case, we should either add it to spec or provide non-normative text advising devs what to do.

Related issues:

There is rough consensus in solid/solid-spec#134 that handling of LDPC resources is an implementation detail.

I've created #109 to provide some answers to the general issue around representation handling.

Regular Accept rule should apply (with optional q parameters) for reading. For writing, it'll fallback on the interaction rules around LDPC specified by applicable methods.

What kind of non-normative text do you think may help? If we can, I prefer to not include such text.

3. "I accidentally created an index.html but forgot to add an .acl, plus that kicks me out of the Data browser, so anyways what do I do now."

I had made a comment about creating a /public/index.html file in a pod on gitter and that it can cause problems for some users so, I wanted to give an update on how to restore your public folder if someone had done this.

Log into your pod, select "your stuff" on the drop down menu, select the "your storage" tab, locate and open the index.html file, click the gear icon and select delete, and your public page will be restored to the default view.
Hope this is helpful.

@dmitrizagidulin

Documenting a clarification on this issue (based on meeting with @dmitrizagidulin ):

If /index.* exists, what happens when / is requested? Common practice: where /index.* placed in the system and / eventually resolves with a message body including the contents of of index.*. There is some overlap with #109 . For example:

must updating /index.* happen through /?

I think we should first have agreement on the interaction, ie. should request to /:

redirect to /index.*
include contents of index.* in the message body
include contents of index.* in the message body and Content-Location: index.* in the header
or something else?

/index.* will inherit the authorization policies from /'s ACL (or ancestor's).

Seen in isolation, I feel that this issue should have been resolved without exposing the existence of index.* to the client at all, it feels much like an filesystemy anachronism, that the use of /index.* is just a way to store a representation of / internally.

The risk that people will use both / and /index.html to refer to the same resource, and therefore complicate URI Normalization is great. That is, it creates a mess for us when we try to determine if two resources are the same, and complicate query and caching.

If we do come up with a consistent way to deal with representations more generally, (#109), then it does make sense to apply that to this problem too. I think it is crucial that the minimal container triples and containment triples are in all representations, and that we minimize the risk that people will use different URIs for what is actually the same resource.

Forgive me if I'm off topic, but I think this is the same issue:

It seems to me like there are 2 types of resources in a Solid pod.

Server/Pod-Provider managed things (e.g. the pod management interface)
User managed things (i.e. LDP resources)

To me, it seems the problems described in this issue are caused by the lack of this distinction in NSS and in Solid in general right now. The Server managed resources should not be able to be managed by the user.

I think this paradigm of separating the User managed from the Server managed is helpful. It could even be advertised by the server. For example, servers could list their server-managed resource(s) (i.e. their "entry point" for users) when receiving a OPTION * request.

@timbl

I discussed this briefly with @timbl , and he said that in the databrowser's use of index.ttl, there was no expectation that it is a default container resource, to the contrary, it is a normal resource, contained as usual.

@timbl

I also discussed index.html briefly with @timbl . First, there is no expectation that all index.* will behave the same. index.ttl is merely a databrowser convention, it is just another resource.

index.html OTOH is a HTML representation of the container, as per old Apache convention. @timbl intended this to be governed by the Accept header.

There are some problems that I see with this approach (that we didn't have time to discuss further), since there are some things that are important in a container's representation, notably the containment triples, the minimal server-managed LDP triples, and the "POSIX metadata", i.e. metadata to indicate ctime, mtime, atime, etc, which would come in handy if Solid was used as an actual file system, e.g. through FUSE.

So, I am uncomfortable about not having them in the representation, even though people could get at them by adjusting the Accept header.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source, so it breaks LDP's model in which LDPC isa LDP-RS.

One possible resolution to this problem is to require the inclusion of the RDF as RDFa. Even though people might PUT index.html without the RDFa, the RDFa could be added to the DOM by the server.

I see three ways to resolve this:

Live with that the HTML representation is not a representation of the container with its triples, but return HTML only if it is requested through the Accept header.
Try to work out something that behaves in the browser like things used to do in the past, but does so in a way that makes it clear that ./index.html is not a representation of ./
Require (or at least say SHOULD) returning the RDF representation of a container as RDFa in the HTML.

My preference would be the latter.

index.html OTOH is a HTML representation of the container

Possibly. To actually resolve that, I was hoping to get an answer to #69 (comment) , #109 (comment) eg. Content-Location?

That aside..

Any representation of a container is required to be RDF bearing; RDF Source; LDP-RS... having triples that can be parsed by an RDF parser... Full stop. Whether it has a certain number of triples or even the "right" triples is orthogonal. That is a different requirement about representations and equivalences we can work out.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source

Not necessarily. If an HTML has no RDFa in it, an RDFa parser will obviously find 0 triples. That's exactly the same as a Turtle, if there are no triples in the document. Attempting to creat or update / with text/turtle or text/html with no RDF can be expected to have the same treatment - basically 0 triples.

#45 (comment) proposes how to handle equivalent representations:

Servers supporting Turtle, JSON-LD and optionally other RDF serializations for LDP-RS SHOULD provide different RDF serializations based on client's proactive negotiation. For example, if a server allows the creation of an LDP-RS in text/html (including RDFa), it SHOULD respond to GET with Accept: text/turtle requests with 200.

@timbl

index.html OTOH is a HTML representation of the container

Possibly. To actually resolve that, I was hoping to get an answer to #69 (comment) , #109 (comment) eg. Content-Location?

Right, so, @timbl saw this as behaving like Apache does, i.e., there's nothing on the surface, the server just silently takes the index.html file on the server and returns it's content as a representation of ./. No Content-Location or anything.

That aside..

Any representation of a container is required to be RDF bearing; RDF Source; LDP-RS... having triples that can be parsed by an RDF parser... Full stop.

That'd be the LDP purist's view, ;-) this stems from the LDP's hierarchy of interaction models, it is not a concern of Solid as currently designed.

Whether it has a certain number of triples or even the "right" triples is orthogonal. That is a different requirement about representations and equivalences we can work out.

Right, but I take the pragmatic view: What are the useful pieces of LDP that we actually need? The mentioned RDF is something we need, and from that, it follows that they must be included with the container representation.

Now, I'm sufficiently purist myself to be wary of a design that doesn't include the container representation with all possible representations, regardless of Accept header. From that, it follows that indeed, any representation is required to be RDF bearing, so we arrive at the same conclusion, but through purism on different parts of the architecture. :-)

However, neither of these views are consistent with the view that @timbl gave me of his motivation behind the index.html, we need to work out something that is.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source

Not necessarily. If an HTML has no RDFa in it, an RDFa parser will obviously find 0 triples. That's exactly the same as a Turtle, if there are no triples in the document. Attempting to creat or update / with text/turtle or text/html with no RDF can be expected to have the same treatment - basically 0 triples.

Right, but my (unstated) assumption is that the HTML would also contain content that is not represented as RDF, that's the whole point of putting HTML there, which makes it non-RDF in LDPs (flawed) model where LDP-RS and LDP-NR are mutually exclusive, regardless of whether there is RDF content hosted in it.

#45 (comment) proposes how to handle equivalent representations:

Servers supporting Turtle, JSON-LD and optionally other RDF serializations for LDP-RS SHOULD provide different RDF serializations based on client's proactive negotiation. For example, if a server allows the creation of an LDP-RS in text/html (including RDFa), it SHOULD respond to GET with Accept: text/turtle requests with 200.

No, that does not appear to address the problem, the point is that the index.html would contain content that is not represented in RDF, that's the whole point, if it was fully represented with RDF, it would not be put as HTML in the first place.

Moreover, the UNIX filesystem analogy extends to containers too, in that the container just contains resources, it is not intended to have an extensive representation on its own.

Edit/Note: I wrote the prior to seeing Kjetil's comment above / in #69 (comment)

This issue was intended to work out whether index.* can be observed or interacted with.

I still think the handling of index.* is ultimately an implementation detail. If it is a container's representation with its own URL, so be it.

#109 (comment) actually tries to resolve this:

where index.* exists by other means, we need to resolve #119

The point of that was to have a clear split between resources that come to existence via Solid's prescribed interaction vs. by other means, and so re "exists by other means" ie. some implementation happens to have index.* with its own accessible URL where it didn't go through an interaction with the container, then it is of course a separate resource (and certainly not a representation of /). Out of Solid spec by default. But the point with 119 is that if there are some unique cases that should have special handling, and so that would help to address:

if /index.* exists, what happens when / is requested, we need to resolve #69

What databrowser, Apache are doing with /index.* are implementation details. So, requesting / should have nothing to do with /index.* because they are different resources. Just because Apache can be configured to handle / and processing/serving /index.* is not something that Solid needs to adopt. Again, if for example a Solid server implementation wants to accept text/html or text/turtle.. at / and store the representations at /index.*, then that's its own decision. It naturally needs to address "if /index.* exists, what happens when / is requested".

I just want to comment very briefly on this:

injection of containment information into HTML is not particularly practical or mature. There is also no implementation experience.

That's wrong. Trellis already supports it as indicated above. We also support it trivially with Perl:

my $gen = RDF::RDFa::Generator->new;
$gen->inject_document($dom, $model);

where $model contains the RDF you wish to inject, and $dom contains the DOM object of the XHTML document you inject into. This is code that has been in production for a decade. I quickhacked a little script that takes an XHTML document on the STDIN and outputs the injected document to STDOUT last night: inject_rdfa.pl.txt

There is plenty of implementation experience, and it is very mature. I would be very surprised if it is not equally simple in JS, it is just about generating the RDF and adding it to the DOM tree, that's all there is to it.

Edit: Didn't notice your comment before sending mine, so here is a quick reply:

That's wrong. Trellis already supports it as indicated above. We also support it trivially with Perl:

It is not an arbitrary injection. Obviously triples can always be thrown in somehow. Even possible with sed but that's probably not a good idea, right? The whole document needs to be properly serialised and be coherent, and anything added or removed from the containment triples should not interfere with everything else, including ideally structure and rendering. Having code to inject is one thing but I'd like to see actual HTML documents in practice that's subject to updates.

In any case, I've noted below that this part of the update is an implementation detail.

Revisiting this.. there is more commonality in the approaches than they seem.

Appending a resource to a container:

PUT /index.html
POST /
Slug: index.html

It should include a triple like:

[about=""]
rel="ldp:contains" href="index.html"

Following are equivalent:

GET /index.html
GET /

Content-Location: index.html

[about=""]
rel="ldp:contains" href="index.html"

[
RDF in HTML best practice:
Don't set base URI in the HTML representation (index.html), but if set, it should end with /. This is so that the RDF graph in index.html is same as /.
]

Is this correct:

[/ or /index.html] MUST then include the RDF representation of the container embedded in the HTML.

Effectively establishes container's HTML to be RDF bearing.

Influences required RDF serializations: #45 ie. adding RDFa (or script with RDF) to Turtle and JSON-LD.

When a new resource is appended or removed from a container, all of its representations (including HTML) needs to incorporate the changes to the containment.

Appending another resource:

POST /

Location: foo

GET /index.html
GET /

Content-Location: index.html

[about=""]
rel="ldp:contains" href="index.html"
rel="ldp:contains" href="foo"

How exactly a server includes the containment triples is an implementation detail. Same level of requirement for Turtle and JSON-LD.

This may be a less of an issue when /index.html is updated directly (eg. PUT /index.html) because the container's integrity falls on the server-imposed constraint ie. listing containment triples.

Server should reject if update to /index.html changes containment triples (aligned with global rule on updating containers). That entails that an RDFa parser is required. Having an RDFa parser makes it possible to serialize to other RDF.

If we treat all representations of a container as equivalent (based on underlying RDF graph), then adding new resources to a container ultimately requires an update to container's description. So the representations need to include the containment triples.

The following criteria can work for updating resources but not sufficient for appending or deleting a resource from a container:

Client ensures that the representation is RDF bearing (in addition to adhering to server constraints) and server verifies.

So, we do need the following any way:

[/ or /index.html] MUST then include the RDF representation of the container embedded in the HTML.

Having said that, there is the question of whether the HTML representation of / may be exempt from this and so a SHOULD instead of MUST. Relaxing the requirement definitely simplifies server and client implementations and allow more flexibility on / (eg. as arbitrary homepage or directory listing).. Keeping it strict means more consistency but also there is a chance that the resulting HTML is not necessarily what the client would like to see/interpret.

TL;DR: Please explain benefit motivating the requirement of embedding RDF in HTML representation of a container.

I honestly don't understand benefit of requiring HTML representation of a container to include RDF (embedded via <script> tag or RDFa). In use case of 'home page', I think user often would want to use some site generator and let it PUT / POST that HTML representing container. If any client needs RDF it can always request text/turlte or application/ld+json. No one stops users to embed RDF in HTML if they have reason for that, still requiring it would force them to regenerate the home page every time they add or remove something from that container. I think that requirement adds burden without any clear benefit, again if someone wants to embed RDF in HTML they can always choose to do it.

There is much repetition at this point. I'm responding below but it'd be great if we can continue the recurring themes and ideas in public chat, calls, F2F etc.

TL;DR: Please explain benefit motivating the requirement of embedding RDF in HTML representation of a container.

The exact same question can be asked for any RDF format. There is no difference between them if the exchange language is RDF and the sources are RDF bearing. Any one can do the job. We can also discuss why HTML+RDFa alone may suffice, and can in fact handle more use cases than the alternatives. It can be very accessible and neither would it require JavaScript to read (or use) a homepage or a directory index. So, why even bother introducing the others? As appealing or appalling as that may be to some, framing as such may not be fruitful.

Once again, if the representations are expected to encode equivalent RDF graphs, then being consistent is a reasonable design decision. The contrary is easy to raise: why should format x and y be expected equivalent but not z, especially when the resource is expected to be RDF bearing with containment information to begin with. This case is not to be conflated with a resource that's deemed to be non-RDF bearing (like an image) and then also providing an RDF-bearing representation. That's not what we want or should be practiced. The homepage case not only predates but is widely deployed than root storage or container. Hence, if the distinction between a homepage and root container is so important, then it only makes sense to leave the homepage or a directory index alone at / and simply use another URI for the root container. Certainly that's not a nice option to some people and it only complicates the situation. So, for the time being, we have to look into how to accommodate both cases - which are actually quite similar - using the same URI.

In use case of 'home page', I think user often would want to use some site generator and let it PUT / POST that HTML representing container.

I am a user. That's not what I want. I do not want to switch between applications just to update different aspects of a resource, or worse, have it switch perspectives based on a particular representation - the classic "pure RDF" and "not so pure RDF". Nothing like that is in practice or can be considered a good design. I want to be able to use an application like dokieli to update / where it works as storage root (including containment information) as well as a human and machine-readable homepage including my WebID Profile. I consider my WebID Profile in a HTML+RDFa to be canonical because that gives the most utility. I want to be able to authenticate using that WebID. If a server can't provide an RDF bearing representation of my WebID Profile or a client unable to parse an RDF bearing representation (as in RDFa), I can't authenticate. Currently, only Turtle (and JSON-LD) are acknowledged by some servers and authentication clients. We need to clarify this gap so that people are not prevented from publishing as they wish while adhering to minimal global requirements.

Proposal for a new solution in #198, which considers “index” representations part of a compound state of a container (and considers the index.html and index.ttl behavior implementation-specific details).

@timbl

clears throat Soooo, since we're not aligning here...

May I just throw out a totally breaking greenfield idea...?

/me types quickly in case anybody was about scream "NOOOOOO!"

Lets make containers server managed, but have it link to any metadata and any other data it might point to for a client to make a reasonable representation of it.

Having containers that have protected data, but also data that can be changed has caused all kinds of problems. Compound state LGTM, but probably not something we can find consensus around.

Then, a more elaborate aux resource system is under consideration, and so, it seems straightforward that a client will pull in data from various sources after GETting a container anyway. Pages built for humans by typical browser UAs tend to consist of a large number of resources anyway.

Interacting directly with it, which was @timbl 's preference can be done trivially if index.html is separate from the container. The difference between @timbl 's proposal and mine is only that previously, the server served index.html, whereas my idea is that index.html is YA auxiliary resource type (and thus isn't necessarily named index.html), is linked from the container and clients will need GET it separately. I acknowledge that this breaks existing clients.

But so much would be simpler if we just made the container server managed and told clients the resources they might want to get for a given application.

@timbl

Interacting directly with it, which was @timbl 's preference can be done trivially if index.html is separate from the container. The difference between @timbl 's proposal and mine is only that previously, the server served index.html, whereas my idea is that index.html is YA auxiliary resource type (and thus isn't necessarily named index.html), is linked from the container and clients will need GET it separately. I acknowledge that this breaks existing clients.

Could we have an exception for GET, where the representation served is the auxiliary resource? But all other interactions need to go through that separate resource?

Could we have an exception for GET, where the representation served is the auxiliary resource? But all other interactions need to go through that separate resource?

We could. We could also say that the container RDF needs to be brought along by injecting RDFa into the resulting representation. But is that exception really worth the bother given that UAs tend to slurp in a large number of resources anyway?

@kjetilk

Lets make containers server managed, but have it link to any metadata and any other data it might point to for a client to make a reasonable representation of it.

Nooooooooooooo (sorry I couldn't help it @kjetilk)

I do have to say that I'm strongly -1 on this approach at the moment, because it would have an immediate breaking impact on a lot of code (including mine), and specifications like shape trees and application interoperability. All of that said, if you provided some concrete examples of what you're proposing, specifically in cases where there is a dependency on and usage of data in the graph of the container resource, I'd be happy to look at ways to reconcile.

Navigation Menu

Repository navigation

Specify/advise semantics of default Container resources (index.html/.ttl) #69

83 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Specify/advise semantics of default Container resources (index.html/.ttl) #69

Description

Activity

csarven commented on Nov 5, 2019

mikeadams1 commented on Nov 16, 2019

csarven commented on Nov 17, 2019

kjetilk commented on Nov 18, 2019

JordanShurmer commented on Nov 27, 2019

kjetilk commented on Dec 5, 2019

kjetilk commented on Dec 9, 2019

csarven commented on Dec 9, 2019

kjetilk commented on Dec 9, 2019

csarven commented on Dec 9, 2019

83 remaining items

kjetilk commented on Feb 3, 2020

csarven commented on Feb 3, 2020

elf-pavlik commented on Feb 4, 2020

csarven commented on Feb 5, 2020

RubenVerborgh commented on Sep 16, 2020

kjetilk commented on Feb 7, 2022

RubenVerborgh commented on Feb 7, 2022

kjetilk commented on Feb 7, 2022

justinwb commented on Feb 7, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions