Skip to content

Feature/webdav trash#35636

Merged
DeepDiver1975 merged 3 commits intomasterfrom
feature/webdav-trash-2
Jul 2, 2019
Merged

Feature/webdav trash#35636
DeepDiver1975 merged 3 commits intomasterfrom
feature/webdav-trash-2

Conversation

@DeepDiver1975
Copy link
Copy Markdown
Member

@DeepDiver1975 DeepDiver1975 commented Jun 24, 2019

Description

We want a WebDAV API as discussed in #15646

This is how it looks like:

  • remote.php/dav/trash-bin is the root folder for all trashbins - each user will find their own trahsbin inside
  • remote.php/dav/trash-bin/$user is the users root for trashbins - it holds all trahsed items of the user listed by file id
  • to restore a trash item use a MOVE operation. In the final implementation the destination can be anything and anywhere: MOVE remote.php/dav/trash-bin/$user/$fileId remote.php/dav/$user/path/to/restore.txt
  • to permanently delete an item from the trashbin perform a DELETE remote.php/dav/trash-bin/$user/$fileId
  • to delete all items from the trashbin perform a DELETE remote.php/dav/trash-bin/$user/

Example Requests

Listing trash items

$ curl 'http://localhost:8080/remote.php/dav/trash-bin/admin//' -X PROPFIND -H 'authorization: Basic ......' -H 'OCS-APIREQUEST: true' -H 'Content-Type: application/xml; charset=UTF-8' -H 'Depth: 1' --data-binary $'<?xml version="1.0"?>\n<d:propfind  xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">\n  <d:prop>\n    <oc:trashbin-original-filename />\n    <oc:trashbin-original-location />\n    <oc:trashbin-delete-timestamp />\n    <d:getcontentlength />\n    <d:resourcetype />\n  </d:prop>\n</d:propfind>' --compressed | xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1817    0  1533  100   284   8612   1595 --:--:-- --:--:-- --:--:-- 10265
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/dav/trash-bin/admin/</d:href>
    <d:propstat>
      <d:prop>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <oc:trashbin-original-filename/>
        <oc:trashbin-original-location/>
        <oc:trashbin-delete-timestamp/>
        <d:getcontentlength/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/trash-bin/admin/2147488234</d:href>
    <d:propstat>
      <d:prop>
        <oc:trashbin-original-filename>Neue Textdatei.txt</oc:trashbin-original-filename>
        <oc:trashbin-original-location>./Neue Textdatei.txt</oc:trashbin-original-location>
        <oc:trashbin-delete-timestamp>1561454917</oc:trashbin-delete-timestamp>
        <d:getcontentlength>17</d:getcontentlength>
        <d:resourcetype/>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/trash-bin/admin/2147488238/</d:href>
    <d:propstat>
      <d:prop>
        <oc:trashbin-original-filename>Neuer Ordner</oc:trashbin-original-filename>
        <oc:trashbin-original-location>./Neuer Ordner</oc:trashbin-original-location>
        <oc:trashbin-delete-timestamp>1561454917</oc:trashbin-delete-timestamp>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:getcontentlength/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

Restoring a file

$ curl 'http://localhost:8080/remote.php/dav/trash-bin/admin/2147488234' -X MOVE -H 'authorization: Basic YWRtaW46YWRtaW4=' -H 'OCS-APIREQUEST: true' -H 'Overwrite: F' -H 'Destination: http://localhost:8080/remote.php/dav/files/admin/test.txt' -H 'X-Request-ID: fcff1fa3-60d3-4b6b-b7bd-06598aeb8b1c' -v 
* Expire in 0 ms for 6 (transfer 0x55816f5d8dd0)
* Expire in 1 ms for 1 (transfer 0x55816f5d8dd0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Expire in 1 ms for 1 (transfer 0x55816f5d8dd0)
* Expire in 1 ms for 1 (transfer 0x55816f5d8dd0)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x55816f5d8dd0)
* Expire in 200 ms for 4 (transfer 0x55816f5d8dd0)
* Connected to localhost (::1) port 8080 (#0)
> MOVE /remote.php/dav/trash-bin/admin/2147488234 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.64.0
> Accept: */*
> authorization: Basic ......
> OCS-APIREQUEST: true
> Overwrite: F
> Destination: http://localhost:8080/remote.php/dav/files/admin/test.txt
> X-Request-ID: fcff1fa3-60d3-4b6b-b7bd-06598aeb8b1c
> 
  0     0    0     0    0     0      0      0 --:--:--  0:00:42 --:--:--     0< HTTP/1.1 201 Created
< Host: localhost:8080
< Date: Tue, 25 Jun 2019 09:39:11 GMT
< Connection: close
< X-Powered-By: PHP/7.2.19-1+0~20190531112545.22+buster~1.gbp75765b
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Set-Cookie: oc_sessionPassphrase=xxxx
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Robots-Tag: none
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Content-Security-Policy: default-src 'none';
< Set-Cookie: xxxx
< Set-Cookie: xxxx
< OC-ETag: "11713caa12b7f721046b55a9d707f86c"
< ETag: "11713caa12b7f721046b55a9d707f86c"
< OC-FileId: 2147488234ocrhbj031xi4
< Content-Length: 0
< Content-type: text/html; charset=UTF-8
< 
  0     0    0     0    0     0      0      0 --:--:--  0:00:42 --:--:--     0
* Closing connection 0

Related Issue

How Has This Been Tested?

  • tests will follow

Screenshots (if appropriate):

Screenshot from 2019-06-21 13-25-17
Screenshot from 2019-06-21 13-25-28
Screenshot from 2019-06-21 13-25-48
Screenshot from 2019-06-21 13-26-48

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Database schema changes (next release will require increase of minor version instead of patch)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised:

Open tasks:

  • Backport (if applicable set "backport-request" label and remove when the backport was done)

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 25, 2019

Codecov Report

Merging #35636 into master will decrease coverage by 0.13%.
The diff coverage is 8.28%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #35636      +/-   ##
============================================
- Coverage     65.68%   65.55%   -0.14%     
- Complexity    18777    18832      +55     
============================================
  Files          1222     1230       +8     
  Lines         70935    71095     +160     
  Branches       1289     1289              
============================================
+ Hits          46597    46603       +6     
- Misses        23960    24114     +154     
  Partials        378      378
Flag Coverage Δ Complexity Δ
#javascript 53.7% <ø> (ø) 0 <ø> (ø) ⬇️
#phpunit 66.9% <8.28%> (-0.16%) 18832 <55> (+55)
Impacted Files Coverage Δ Complexity Δ
apps/dav/lib/TrashBin/TrashBinFolder.php 0% <0%> (ø) 8 <8> (?)
apps/dav/lib/Connector/Sabre/Directory.php 70.12% <0%> (-0.87%) 75 <0> (+1)
apps/dav/lib/TrashBin/AbstractTrashBinNode.php 0% <0%> (ø) 13 <13> (?)
apps/dav/lib/TrashBin/TrashBinManager.php 0% <0%> (ø) 12 <12> (?)
apps/dav/lib/TrashBin/TrashBinHome.php 0% <0%> (ø) 6 <6> (?)
apps/dav/lib/TrashBin/TrashBinItemsFolder.php 0% <0%> (ø) 5 <5> (?)
apps/dav/lib/Capabilities.php 0% <0%> (ø) 3 <0> (ø) ⬇️
apps/dav/lib/TrashBin/TrashBinFile.php 0% <0%> (ø) 3 <3> (?)
apps/dav/lib/RootCollection.php 100% <100%> (ø) 1 <0> (ø) ⬇️
apps/dav/lib/TrashBin/TrashBinPlugin.php 20% <20%> (ø) 6 <6> (?)
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d356f5...c06435e. Read the comment docs.

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 25, 2019

Codecov Report

Merging #35636 into master will increase coverage by 0.03%.
The diff coverage is 92.53%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #35636      +/-   ##
============================================
+ Coverage     65.77%   65.81%   +0.03%     
- Complexity    18776    18818      +42     
============================================
  Files          1222     1228       +6     
  Lines         70871    70982     +111     
  Branches       1289     1289              
============================================
+ Hits          46617    46716      +99     
- Misses        23876    23888      +12     
  Partials        378      378
Flag Coverage Δ Complexity Δ
#javascript 53.7% <ø> (ø) 0 <ø> (ø) ⬇️
#phpunit 67.19% <92.53%> (+0.03%) 18818 <60> (+42) ⬆️
Impacted Files Coverage Δ Complexity Δ
apps/dav/lib/Connector/Sabre/Directory.php 70.83% <ø> (-0.86%) 77 <0> (+1)
apps/dav/lib/Capabilities.php 0% <0%> (ø) 3 <0> (ø) ⬇️
apps/dav/lib/Connector/Sabre/FilesPlugin.php 87.07% <100%> (ø) 77 <2> (ø) ⬇️
apps/dav/lib/RootCollection.php 100% <100%> (ø) 1 <0> (ø) ⬇️
apps/dav/lib/TrashBin/TrashBinHome.php 100% <100%> (ø) 7 <7> (?)
apps/dav/lib/TrashBin/RootCollection.php 100% <100%> (ø) 2 <2> (?)
apps/files_trashbin/lib/Trashbin.php 77.8% <100%> (+0.09%) 147 <18> (+1) ⬆️
apps/dav/lib/Files/BrowserErrorPagePlugin.php 90% <100%> (ø) 5 <2> (ø) ⬇️
apps/dav/lib/TrashBin/TrashBinFile.php 100% <100%> (ø) 2 <2> (?)
apps/dav/lib/Server.php 50.67% <50%> (+0.33%) 24 <0> (ø) ⬇️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dad0eb5...c0882c3. Read the comment docs.

@felix-schwarz
Copy link
Copy Markdown

felix-schwarz commented Jun 25, 2019

  • remote.php/dav/trash-bin/admin is the users root for trashbins - it holds all trahsed items listed by file id

What would the response to a PROPFIND on that location look like? Which metadata would be available?

Would the remote.php/dav/trash-bin/admin root allow for change detection via ETag (just like the regular root folder does)?

  • to restore a trash item use a MOVE operation. In the final implementation the destination can be anything and anywhere: MOVE remote.php/dav/trash-bin/$user/$fileId remote.php/dav/$user/path/to/restore.txt

How can a client implement functionality identical to the Restore button in the current web UI?

The previous PR mentioned a special folder to MOVE to to achieve this. Is this still planned?

If not (or alternatively), providing the original location as part of the response to a PROPFIND on remote.php/dav/trash-bin/admin would have the benefit that clients could show users where an item will be restored to (and possibly navigate there following a restore).

@DeepDiver1975 @davigonz @hosy @guruz @ckamm @ogoffart

@DeepDiver1975
Copy link
Copy Markdown
Member Author

@felix-schwarz THX for the feedback - I'll update the description asap.

One thing is clear: there is no etag propagation on the trashbin. I agree this would be nice but it is not possible.
What is also missing on the trash-bin endpoint is the capability to load page-wise (like the REPORT on the files endpoint)

@davigonz
Copy link
Copy Markdown

to restore a trash item use a MOVE operation. In the final implementation the destination can be anything and anywhere: MOVE remote.php/dav/trash-bin/$user/$fileId remote.php/dav/$user/path/to/restore.txt

Yes, this solution is much better than the previous PR one, as long as you mean moving the restored file to its previous location when you say "the destination can be anything and anywhere". There should not be a new location for a restored file, but the original one.

remote.php/dav/trash-bin/admin is the users root for trashbins - it holds all trashed items listed by file id

What is the purpose of this API? I do not see a usecase in which admin users would like to restore normal user files, unless restore option can be disabled for normal users. It's like if we had remote.php/dav/files/admin to see all the user files, it sounds a bit weird to me.

If not (or alternatively), providing the original location as part of the response to a PROPFIND on remote.php/dav/trash-bin/admin would have the benefit that clients could show users where an item will be restored to (and possibly navigate there following a restore).

Not sure if this would be needed, a restored file should be moved to its previous location, not anywhere and that location could keep inmutable in local database path field.

@DeepDiver1975
Copy link
Copy Markdown
Member Author

Yes, this solution is much better than the previous PR one, as long as you mean moving the restored file to its previous location when you say "the destination can be anything and anywhere". There should not be a new location for a restored file, but the original one.

The client has to grab the original location from the webdav property (see the screenshots - which I will update asap). But in addition the api allows to restore the file to any other location - e.g. if there is already a file with the same name living in the original location.

What is the purpose of this API? I do not see a usecase in which admin users would like to restore normal user files, unless restore option can be disabled for normal users. It's like if we had remote.php/dav/files/admin to see all the user files, it sounds a bit weird to me.

Maybe a bit missleading descrpiption ...
remote.php/dav/trash-bin/admin/ holds the trash items of admin
remote.php/dav/trash-bin/tom/ holds the trash items of tom

@DeepDiver1975
Copy link
Copy Markdown
Member Author

(example curl commands for listing trash items and for restoring have been added to the description)

@davigonz
Copy link
Copy Markdown

davigonz commented Jun 25, 2019

Thanks for the explanation @DeepDiver1975

The client has to grab the original location from the webdav property (see the screenshots - which I will update asap). But in addition the api allows to restore the file to any other location - e.g. if there is already a file with the same name living in the original location.

I see your point, what I'm used to see when there's already a file with the same name living in the original location is:

  1. File is automatically renamed to something like file(restored).txt
  2. A dialog with the options Keep both and Replace is prompted.

Both scenarios end up with the file restored in the same location. The other approach is more flexible for sure but in my opinion doesn't fully fit with the definition for restoring a deleted file. Besides, it would require more effort in the clients since the users would need to choose the new location for the file and if there's a conflict with an existing file there, ask them to keep both or replace.

@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch from c06435e to dc9f6e1 Compare June 25, 2019 11:15
@DeepDiver1975
Copy link
Copy Markdown
Member Author

Changing the Destination in server Side violates the move semantics.
Furthermore i consider more flexibility a Bonus.

@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch 2 times, most recently from 4cd9834 to 215db92 Compare June 25, 2019 14:08
Copy link
Copy Markdown
Contributor

@butonic butonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added flexibility is a bonus:

  • clients can show the origin of a deleted file (@deepdiver do we have a path in addition to the parent fileid? the path to the original location might have changed) at the cost of doing the lookup every time they want to restore a file with a MOVE
  • the restore target path determination logic moves to the client, which is more more complex to handle, but relieves the server from a lot of special purpose code and simplifies the API because we use a standard MOVE to restore the file, no magic source or target node

@DeepDiver1975
Copy link
Copy Markdown
Member Author

* (@deepdiver do we have a path in addition to the parent fileid? the path to the original location might have changed)

trash items have 2 webdav properties for these cases:

Looking at them they seem to have some redundency. {http://owncloud.org/ns}trashbin-original-location might be enough

@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch from 215db92 to 123f6bf Compare June 26, 2019 07:28
@davigonz
Copy link
Copy Markdown

davigonz commented Jun 26, 2019

Changing the Destination in server Side violates the move semantics.

Agreed, what I meant is that using the MOVE operation is fine but it does not necessarily mean that the restore target path needs to be chosen by the final user.

I was thinking in this flow:

  1. Admin user selects the deleted files option:
  2. Admin user wants to restore picture.png, located in root directory:
  3. If the file already exists in the restore target path, let the user decides what to do with it => Keep both, replace or nothing.

clients can show the origin of a deleted file (@deepdiver do we have a path in addition to the parent fileid? the path to the original location might have changed) at the cost of doing the lookup every time they want to restore a file with a MOVE

I think that showing the origin of a deleted file is possible even without the flexibility of letting the user choose the final destination of the file.

the restore target path determination logic moves to the client, which is more more complex to handle, but relieves the server from a lot of special purpose code and simplifies the API because we use a standard MOVE to restore the file, no magic source or target node

In case you want to proceed with the MOVE operation to anywhere, yes, it makes total sense to handle that logic in the client.

Just sharing my thoughts, proceed with the solution you consider better 😉

@butonic
Copy link
Copy Markdown
Contributor

butonic commented Jun 26, 2019

@davigonz yap, for now the clients should just try to restore to the original folder (id based). If that fails for whatever reason the client can either follow the old implementation and append the (restored), although that might run into filename lenght issues, or present the user with a file picker dialog ...

In any case, I think this actually makes it easier to control what is happening. The less magic the better.

@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch 2 times, most recently from 8ade5af to 3859f02 Compare June 26, 2019 11:38
@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch 3 times, most recently from 9a08115 to ca1498d Compare June 27, 2019 10:11
@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch 3 times, most recently from 2b62937 to 5e78b7e Compare June 28, 2019 07:35
Copy link
Copy Markdown
Member

@individual-it individual-it left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the webdav API we should not need And user "user0" has logged in to a web-style session in the tests

| old |
| new |

Scenario Outline: A file deleted from a folder is restored to root if the original folder does not exist
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you had to remove that scenario? Is it not applicable anymore?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - the behavior of restore changed with the new API. The target location is fully controlled by the clients. There is no need in verifying that under certain situations the file is restore into a different location.

@individual-it individual-it requested a review from phil-davis June 28, 2019 08:38
@DeepDiver1975 DeepDiver1975 force-pushed the feature/webdav-trash-2 branch from 5e78b7e to 252d899 Compare June 28, 2019 14:13
@felix-schwarz
Copy link
Copy Markdown

Would it be possible to compute a hash over the largest oc:trashbin-delete-timestamp value and the total number of items (both files and collections) in the trash (f.ex. sha1("1561454917"."2") for the example above) - and then return it as ETag for the trash root folder?

Then a PROPFIND with Depth: 0 could be used to check for changes.

@DeepDiver1975 @davigonz @hosy @guruz @ckamm @ogoffart

@DeepDiver1975
Copy link
Copy Markdown
Member Author

with the webdav API we should not need And user "user0" has logged in to a web-style session in the tests

@individual-it I assume this can be done all over the place in all feature files where the trashbin api is used in steps

@DeepDiver1975
Copy link
Copy Markdown
Member Author

Would it be possible to compute a hash over the largest oc:trashbin-delete-timestamp value and the total number of items (both files and collections) in the trash (f.ex. sha1("1561454917"."2") for the example above) - and then return it as ETag for the trash root folder?

Then a PROPFIND with Depth: 0 could be used to check for changes.

@DeepDiver1975 @davigonz @hosy @guruz @ckamm @ogoffart

@felix-schwarz let me see what I can do ....

Copy link
Copy Markdown
Contributor

@butonic butonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same date format as for getlastmodified.

@DeepDiver1975 DeepDiver1975 merged commit 27d12b0 into master Jul 2, 2019
@delete-merged-branch delete-merged-branch bot deleted the feature/webdav-trash-2 branch July 2, 2019 05:47
@phil-davis
Copy link
Copy Markdown
Contributor

Backport stable10 #35716

@DeepDiver1975 DeepDiver1975 changed the title Feature/webdav trash 2 Feature/webdav trash Jul 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants