[QA] occ upgrade does nothing, although web-ui says 'Update needed'

[jnweiger]

Seen in 10.7.0 RC1

I have no clear reproducer for this, but I ran by accdent into this situation multiple times:

The web ui asks for update.

The occ command does not work: BAD

occ upgrade   
ownCloud is already latest version

Attempted workaround:

• in config.php switch upgrade.disable-web' => false,
• a new error occurs:
  It looks like your instance may host many files and/or users. To ensure a smooth upgrade process, please use the command line updater (occ upgrade)

This is a deadlock situation.

Evil workaround:

• hack into "lib/base.php" and force $tooBig = false; after the ldap and user evaluation code and before the error printing.
• now the web interface allows to run the upgrade procedure. Login is possible again.

Suggested improvement:
Add a --force option to occ upgrade to try as hard as the web version would do.

[micbar]

I have no clear reproducer for this, but I ran by accdent into this situation multiple times:

we need that context, otherwise we cannot qualify

[jnweiger]

I have no clear reproducer for this, but I ran by accdent into this situation multiple times:

we need that context, otherwise we cannot qualify

One instance, where this happened repeatedly yesterday and today is still online.
All I can do for now, is leave that instance alone, and give investigators ssh access.

[cortho]

copied from #37361

Still having this issue with a recent ownCloud 10.7

The expected behavior is to have two app versions in two folders, when updating an app that is installed in the read-only app folder?

yes, 'writable' => false, means oC has no permissions to manage files and directories in this app folder so deletion of the previous app version is not allowed.

While I agree with this behavior I think the ownCloud app loading mechanism is broken. I did not dig into the code but shouldn't ownCloud browse all available apps and then use the most recent version, regardless of their locations?

Since it is the default ownCloud behavior to come with a read only apps directory and a writable apps-external directory it must not be delegated to the end user managing apps versions.

I consider this a nasty bug which leads to lots of confusion, especially for beginners. See some postings from central as reference:

• https://central.owncloud.org/t/disable-update-needed-message/32481
• https://central.owncloud.org/t/automatic-updating-is-disabled/32869
• https://central.owncloud.org/t/update-needed-and-nobody/32982
• https://central.owncloud.org/t/update-needed-although-up-to-date/29413
• https://central.owncloud.org/t/internal-server-error-owncloud-10-7-0/33052
• https://central.owncloud.org/t/server-version-10-5-0-10-requests-an-upgrade-over-and-over-again/30847

and so on ...

Edit: reproducable with any owncloud-complete installation where the market app updated any app (activity, files_pdfviewer, ...) using the default ro apps and rw apps-external directories

[cortho]

@micbar do you need more context or is the above information sufficent for you?

[micbar]

@cortho Thanks. I will add it to the support queue.

[JammingBen]

To understand the whole problem, I digged a bit into the whole app loading process. When loading (not updating!) an app, OC will use the latest app in the apps_paths array in the config. Meaning, if you have something like:

'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/oc/projects/owncloud/core/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/oc/projects/owncloud/core/apps-external',
      'url' => '/apps-external',
      'writable' => true,
    ),
  ),

ownCloud will prioritize the app located in apps-external. This happens because the code simply travels through the array and inserts the result into the cache. Let's say you have the activity app in both dirs. ownCloud will then go though the array, first setting $availableApps['activity'] = '.../apps' and then overwriting it with $availableApps['activity'] = '.../apps-external'. As I said, this will be stored in the cache.

@VicDeo As you've dealt with this topic in the past: Do you know if this the intended behavior? I'm just wondering, because on the other hand,findAppInDirectories() (https://github.com/owncloud/core/blob/master/lib/private/App/AppManager.php#L697) will return the version and path for the newest version - regardless of the array order. I might be overseeing something, but to me it looks like we have an inconsistency here.

However, I was not able to reproduce the original issue from Jürgen yet. And I can't say for sure how these things are connected.

[jvillafanez]

When loading (not updating!) an app, OC will use the latest app in the apps_paths array in the config.

That's definitely wrong. There is no guarantee that folder will contain the latest app version. Consider the following scenario.

1. Install OC 10.x. package. The package contains a predefined user_ldap app
2. Update the user_ldap app to the latest available version. Now, the "/apps" folder contains user_ldap version 1 and the "/files_external" the version 2. Note that this isn't the latest version available due to core restrictions (latest version might require a newer core)
3. Update OC to the latest version. The package contains a user_ldap version 3 (not version 2)

In that case, you have a situation where the core has a newer version of the app.

The other option is that the entries in the config.php have been swapped, but that would be weird. Following this option, we could have 2 causes, both of them unlikely:

• An external (non-PHP) script creates those entries. Maps / dictionaries aren't sorted, which means that even if you enter A B and C, you could traverse the items as B C A. In any case PHP guarantees that the insertion order is preserved (as far as I know), and the code seems to be generated by PHP
• The admin edited the file. I don't think admins would need to touch this configuration, so this is also unlikely.

It seems the upgrade route is the most plausible explanation.

[JammingBen]

In that case, you have a situation where the core has a newer version of the app.

Absolutely right. But it least for me, OC will still load the version 2 app from apps-external. Can you confirm this behavior in general?

[jvillafanez]

It seems the problem is in

core/lib/private/App/AppManager.php

Lines 513 to 515 in 817f54f

	// cache results for a day
	$this->appInfo->set($appId, $data, 86400);
	$this->appInfo->set($file, $data, 86400);

We have 2 apps with the same file id, but the second has a lower version. The data cached by filename should be the right one, but there are several methods accessing by appid, which gets overwritten with the data from the second (lower) version.

If Jürgen's instance is still up and running, we could try to remove the line 514 and check if it fixes the problem. This is mainly to verify the problem is there, but we'll still need to fix the caching. It should be possible to compare the versions of the cached data and the new data in order to decide whether we should overwrite or not.

[JammingBen]

Yep, that matches with my findings.

If Jürgen's instance is still up and running, we could try to remove the line 514 and check if it fixes the problem. This is mainly to verify the problem is there, but we'll still need to fix the caching. It should be possible to compare the versions of the cached data and the new data in order to decide whether we should overwrite or not.

I already asked him, unfortunately the instance is not available anymore.

[jvillafanez]

It might be related... I've checked with user_ldap 0.15.4 in the apps folder and 0.14.0 in the apps-external folder. This seems to work although I have

  - user_ldap:
    - Version: 0.14.0
    - Path: /var/www/owncloud/apps/user_ldap

The inconsistency is there, but it doesn't seem to trigger the "update needed" in the web UI

[JammingBen]

The inconsistency is there, but it doesn't seem to trigger the "update needed" in the web UI

I'm also unable to reproduce the issue. However, maybe the affected instances somehow corrupted their cache, which leads to problems because if this inconsistency. We definitely need an instance to reproduce this error. I also pinged @enbrnz in chat because he experienced this issue in the past apparently, no answer yet though.

[jvillafanez]

I might have something to reproduce the issue.
The key point is that both paths must be writable and having the app in both locations. This causes the market app to put the updated app in the first location while the info is picked from the second. This causes a faulty app upgrade in the sense that the installed version stored in the DB to be kept as the old one, which causes the next upgrade notifications.

Steps roughly should be as follows:

1. Start copying an app from the apps folder (first location) to the custom folder (second location). This should simulate a situation where an app has been updated to the latest available version.
2. Set the apps folder as writable in the config.php
3. update the app via occ market:upgrade

So we should have reached the point we're the DB has the "old" installed_version for the app, but there is an updated version already installed.

The local cache (ACPu) seems to be part of the problem. Cleaning the local cache (and likely when the cache expires after 1 day) fixes the "update needed" loop.

[VicDeo]

so the solution will be to clean up the app cache on upgrade.