[For 10.4][API] user-sync OCS API#36428

micbar · 2019-11-13T19:16:13Z

Description

External user provisioning system need an http api to trigger user-sync for a specific user.

Authorization

The http API can only be executed by a user with admin priviledges. Suggestion is to create a technical user who is in the admin group

Route

 curl -X POST http://localhost:8080/ocs/v2.php/cloud/user-sync/admin -v -u admin

Response

200 - if sync was executed
404 - given userId is unknown
409 - mutliple users have been found for the given user id - not unique user criteria

Related Issue

refs https://github.com/owncloud/enterprise/issues/3521

How Has This Been Tested?

user curl command as above with an admin user

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Database schema changes (next release will require increase of minor version instead of patch)
Breaking change (fix or feature that would cause existing functionality to change)
Technical debt
Tests only (no source changes)

Checklist:

Code changes
Unit tests added
Acceptance tests added
Documentation ticket raised:

Open tasks:

Changelog item, see TEMPLATE

micbar · 2019-11-13T19:18:02Z

@phil-davis Now you can add the acceptance tests to this branch.

This is a breaking change for the next minor version.
We will do the full QA here before merging.

dpakach · 2019-11-14T06:40:45Z

@micbar @DeepDiver1975 , I have added acceptance tests for checking the cases when there are not any external user backends. Some status codes seem out of place. Please have a look

phil-davis · 2019-12-02T06:04:12Z

@micbar @DeepDiver1975 , I have added acceptance tests for checking the cases when there are not any external user backends. Some status codes seem out of place. Please have a look

@micbar @DeepDiver1975 please have a look at the added acceptance test scenarios and see if any of the failure codes need to be adjusted/made more consistent.

phil-davis · 2019-12-02T06:06:37Z

@dpakach I don't think we ever sorted out how to add a "happy path" acceptance test for this in the user_ldap tests. Can link post here about how far you got (or link to a comment somewhere)?

We really need to have a "happy path" test passing somewhere.

dpakach · 2019-12-02T06:41:25Z

@dpakach I don't think we ever sorted out how to add a "happy path" acceptance test for this in the user_ldap tests. Can link post here about how far you got (or link to a comment somewhere)?

We really need to have a "happy path" test passing somewhere.

@phil-davis I tried doing that but could not make work. Either there is some problem in my configuration or some bug. I had created the issue #36312.
I think those tests should be added in user_ldap.

DeepDiver1975 · 2019-12-02T08:27:59Z

@micbar @DeepDiver1975 please have a look at the added acceptance test scenarios and see if any of the failure codes need to be adjusted/made more consistent.

I'm not that deep into the acceptance tests these days - I see you as the lead dev on these topic @phil-davis .

Regarding the tests not working properly I cannot add more then I did on #36312
We have this API live in production for weeks now and it works - with an ldap backend.

phil-davis · 2019-12-02T08:40:05Z

Regarding the tests not working properly I cannot add more then I did

NP - I will look with @dpakach and our guys

codecov · 2019-12-03T04:40:20Z

Codecov Report

Merging #36428 into master will decrease coverage by <.01%.
The diff coverage is 70.96%.

@@             Coverage Diff              @@
##             master   #36428      +/-   ##
============================================
- Coverage     64.68%   64.67%   -0.01%     
- Complexity    19071    19106      +35     
============================================
  Files          1268     1269       +1     
  Lines         74516    74723     +207     
  Branches       1311     1320       +9     
============================================
+ Hits          48201    48329     +128     
- Misses        25929    26006      +77     
- Partials        386      388       +2

Flag	Coverage Δ	Complexity Δ
#javascript	`54.12% <ø> (+0.1%)`	`0 <ø> (ø)`	⬇️
#phpunit	`65.84% <70.96%> (-0.02%)`	`19106 <9> (+35)`

Impacted Files	Coverage Δ	Complexity Δ
core/routes.php	`48.71% <ø> (ø)`	`0 <0> (ø)`	⬇️
core/Application.php	`23.57% <10%> (-1.21%)`	`1 <0> (ø)`
lib/private/User/SyncService.php	`83.72% <100%> (+0.09%)`	`61 <4> (+1)`	⬆️
core/Controller/UserSyncController.php	`100% <100%> (ø)`	`5 <5> (?)`
...amework/Middleware/Security/SecurityMiddleware.php	`95.16% <100%> (+1.82%)`	`24 <0> (+1)`	⬆️
core/Command/User/SyncBackend.php	`78.89% <100%> (ø)`	`38 <0> (ø)`	⬇️
core/Command/App/Enable.php	`87.87% <0%> (-12.13%)`	`8% <0%> (+3%)`
core/js/sharedialogshareelistview.js	`73.23% <0%> (-7.91%)`	`0% <0%> (ø)`
lib/private/Share20/ProviderFactory.php	`85.36% <0%> (-6.74%)`	`17% <0%> (+1%)`
apps/files_sharing/lib/Capabilities.php	`93.15% <0%> (-5.19%)`	`12% <0%> (+2%)`
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d74d406...dcacc2a. Read the comment docs.

tests/acceptance/features/apiMain/userSync.feature

DeepDiver1975 · 2019-12-05T13:10:13Z

Feel free to fix right away, Phil

phil-davis · 2019-12-05T14:25:17Z

Note: I rebased just now so that we have this on top of the latest master.

phil-davis · 2019-12-06T06:35:30Z

Feel free to fix right away, Phil

@micbar I have lots to do, and I do not see easily how to fix/catch this 500 status code.

NotAdminException is thrown - I guess in SecurityMiddleware.php and that should be caught somewhere and turned into a nicer HTTP status like 403.

Strangely, I do not see other examples of places where it is currently being caught.

Some developer will know what to do in 5 minutes (maybe)?

micbar · 2019-12-06T12:06:44Z

Thanks @sharidas . Let us see what CI says.

micbar · 2019-12-06T12:07:10Z

@phil-davis @skshetry The response Code is now 403 Forbidden.

phil-davis · 2019-12-06T12:21:10Z

@phil-davis @skshetry The response Code is now 403 Forbidden.

I suspect that this will be a more general fix that might catch other places where we test sending an API request to an admin-only endpoint, but with only authentication of an ordinary user. Probably the status codes will now be better. There might be some test expectations to adjust (improve) when we see the CI result.

phil-davis · 2019-12-17T04:15:15Z

I rebased just now to make sure this is up-to-date and passing.

Fix the status code for non admin access Signed-off-by: Sujith H <sharidasan@owncloud.com>

comments were addressed

phil-davis

Tests are OK here, as far as they go.
We will need to work out a way to do a manual test of this with LDAP set-up, because I never managed to get the user_ldap CI setup to successfully do a user sync through this API.
Someone else needs to approve the actual code.

VicDeo

Looks good, just some minor changes

lib/private/User/SyncService.php

core/Controller/UserSyncController.php

lib/private/AppFramework/Middleware/Security/SecurityMiddleware.php

jvillafanez · 2019-12-19T13:13:34Z

core/Controller/UserSyncController.php

+	 */
+	public function syncUser($userId): Result {
+		foreach ($this->userManager->getBackends() as $backEnd) {
+			$users = $backEnd->getUsers($userId, 2);


If I remember correctly, we had already problems if there are users with the similar name. What if we have "Scott", "Scotty", "Scott_A"... and we want to sync "Scott"? Even if we have only one "Scott" in the system, I think the method will return an error.

You mean it will throw an exception? Or return an empty value?

core/Controller/UserSyncController.php

jvillafanez · 2019-12-19T13:31:44Z

lib/private/AppFramework/Middleware/Security/SecurityMiddleware.php

 			if ($exception instanceof NotLoggedInException) {
 				return $controller->buildResponse(new Result(null, API::RESPOND_UNAUTHORISED, 'Unauthorised'));
+			} elseif ($exception instanceof  NotAdminException) {
+				return $controller->buildResponse(new Result(null, $exception->getCode(), $exception->getMessage()));


The exception doesn't explicitly says that the error code should be an HTTP code, and it doesn't guarantee it. We should at least include a comment saying that we expect an HTTP code as the error code of the exception.

The docblock of the parent method mentions that the return value is a http Response object. Isn't that sufficient?

NotAdminException has a code Http::STATUS_FORBIDDEN so it looks ok from my side...
https://github.com/owncloud/core/blob/master/lib/private/AppFramework/Middleware/Security/Exceptions/NotAdminException.php#L37

phil-davis · 2020-05-26T08:31:31Z

Note: acceptance test added in owncloud/user_ldap#562

micbar self-assigned this Nov 13, 2019

micbar added 2 - Developing dev:acceptance-tests labels Nov 13, 2019

micbar added this to the QA milestone Nov 13, 2019

micbar added the QA:team label Nov 13, 2019

micbar changed the base branch from feature/user-sync to master November 13, 2019 19:19

micbar assigned phil-davis and unassigned micbar Nov 13, 2019

micbar changed the title ~~[API] user-sync OCS API~~ [New Minor Version][API] user-sync OCS API Nov 13, 2019

phil-davis assigned dpakach and unassigned phil-davis Nov 14, 2019

dpakach removed their assignment Nov 20, 2019

dpakach mentioned this pull request Nov 20, 2019

Add acceptance tests for user-sync OCS API #36236

Open

micbar mentioned this pull request Nov 21, 2019

Document Single User Sync owncloud/docs#2105

Closed

12 tasks

micbar added the documentation label Nov 21, 2019

micbar changed the title ~~[New Minor Version][API] user-sync OCS API~~ [For 10.4][API] user-sync OCS API Nov 25, 2019

micbar added the minor-release label Nov 25, 2019

phil-davis self-assigned this Dec 3, 2019

phil-davis force-pushed the feature/user-sync-2 branch from 27e40ae to 4bd54cc Compare December 3, 2019 04:39

phil-davis reviewed Dec 3, 2019

View reviewed changes

tests/acceptance/features/apiMain/userSync.feature Show resolved Hide resolved

tests/acceptance/features/apiMain/userSync.feature Outdated Show resolved Hide resolved

DeepDiver1975 previously requested changes Dec 4, 2019

View reviewed changes

tests/acceptance/features/apiMain/userSync.feature Outdated Show resolved Hide resolved

tests/acceptance/features/apiMain/userSync.feature Show resolved Hide resolved

phil-davis mentioned this pull request Dec 5, 2019

Tests against core test-user-sync-api owncloud/user_ldap#480

Closed

phil-davis force-pushed the feature/user-sync-2 branch from 4bd54cc to 548d43a Compare December 5, 2019 14:24

sharidas force-pushed the feature/user-sync-2 branch 4 times, most recently from 8589764 to 3696eca Compare December 9, 2019 07:26

phil-davis force-pushed the feature/user-sync-2 branch from 3696eca to a9fc898 Compare December 17, 2019 04:14

phil-davis requested a review from DeepDiver1975 December 17, 2019 04:15

DeepDiver1975 and others added 3 commits December 19, 2019 13:00

[API] user-sync OCS API

5465515

Add tests to check user-sync api with any external user backend

69c5732

Fix the status code for non admin access

2a11200

Fix the status code for non admin access Signed-off-by: Sujith H <sharidasan@owncloud.com>

phil-davis force-pushed the feature/user-sync-2 branch from a9fc898 to 2a11200 Compare December 19, 2019 07:15

phil-davis reviewed Dec 19, 2019

View reviewed changes

VicDeo requested changes Dec 19, 2019

View reviewed changes

jvillafanez reviewed Dec 19, 2019

View reviewed changes

Improvements from code review

dcacc2a

VicDeo approved these changes Dec 20, 2019

View reviewed changes

micbar merged commit afb447b into master Dec 20, 2019

delete-merged-branch bot deleted the feature/user-sync-2 branch December 20, 2019 11:57

This was referenced May 25, 2020

user_ldap acceptance tests failing #37428

Closed

User sync with API owncloud/user_ldap#562

Merged

Conversation

micbar commented Nov 13, 2019

Description

Authorization

Route

Response

Related Issue

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Open tasks:

Uh oh!

micbar commented Nov 13, 2019

Uh oh!

dpakach commented Nov 14, 2019

Uh oh!

phil-davis commented Dec 2, 2019

Uh oh!

phil-davis commented Dec 2, 2019

Uh oh!

dpakach commented Dec 2, 2019

Uh oh!

DeepDiver1975 commented Dec 2, 2019

Uh oh!

phil-davis commented Dec 2, 2019 • edited by DeepDiver1975 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Dec 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DeepDiver1975 commented Dec 5, 2019

Uh oh!

phil-davis commented Dec 5, 2019

Uh oh!

phil-davis commented Dec 6, 2019

Uh oh!

micbar commented Dec 6, 2019

Uh oh!

micbar commented Dec 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phil-davis commented Dec 6, 2019

Uh oh!

phil-davis commented Dec 17, 2019

Uh oh!

phil-davis left a comment

Choose a reason for hiding this comment

Uh oh!

VicDeo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jvillafanez Dec 19, 2019

Choose a reason for hiding this comment

Uh oh!

micbar Dec 19, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jvillafanez Dec 19, 2019

Choose a reason for hiding this comment

Uh oh!

micbar Dec 19, 2019

Choose a reason for hiding this comment

Uh oh!

VicDeo Dec 20, 2019

Choose a reason for hiding this comment

Uh oh!

phil-davis commented May 26, 2020

phil-davis commented Dec 2, 2019 •

edited by DeepDiver1975

Loading

codecov bot commented Dec 3, 2019 •

edited

Loading

micbar commented Dec 6, 2019 •

edited

Loading