Static Caching invalidator not working when running from the queue

[jameswtc]

Bug Description

The static cache file not removed when the queue is run through php artisan queue:work, (or using database queue).

After hours of inspection, the is due to the caching of full URL in the cache:

the problem is in /cms/src/StaticCaching/Cachers/FileCacher.php
line 36

$this->getUrl($request)

in /cms/src/StaticCaching/StaticCacheManager.php
line 49

'base_url' => $this->app['request']->root(),

The request object is not available when running the command from Laravel artisan, and the root url resolved to localhost and the scheme is set to http instead of https.

This caused the $this->getUrl($request) not able to get the cached version of the url with the actual host.

How to Reproduce

In the .env file, set

CACHE_STRATEGY=full
...
QUEUE_CONNECTION=database

Setup the queue worker to process the queue.

Modify a collection entry and save it.

Inspect the static folder, the cached file still presents, i.e., not removed by invalidator.

Environment

Statamic version: 3.0.39

PHP version: 7.4

FIX:

I managed to make it work by reading the base_url from the .env file instead of resolving it from the request object.

      return array_merge($config, [
          'exclude' => $this->app['config']['statamic.static_caching.exclude'] ?? [],
          'ignore_query_strings' => $this->app['config']['statamic.static_caching.ignore_query_strings'] ?? false,
          'base_url' => env('BASE_URL'), // $this->app['request']->root(),
          'locale' => Site::current()->handle(),
      ]);

[edalzell]

We are running half-measure static caching and running into something similar. We are using the queue (Redis) cuz we have the git integration turned on.

[edalzell]

I tried 'base_url' => URL::getSiteUrl(),, but that did not fix this issue, for me.

[jameswtc]

I tried 'base_url' => URL::getSiteUrl(),, but that did not fix this issue, for me.

For this to work, the URL must also be cached in your chosen caching driver too. If for some reason you clear your cache, then you have to manually remove it too.

The StaticCacheManager first check for the URL in the cache to retrieve the file path.

[edalzell]

I tried 'base_url' => URL::getSiteUrl(),, but that did not fix this issue, for me.

For this to work, the URL must also be cached in your chosen caching driver too. If for some reason you clear your cache, then you have to manually remove it too.

The StaticCacheManager first check for the URL in the cache to retrieve the file path.

@jameswtc I don't quite understand this, can you explain what you mean? I'd like to make a PR, as this still affects us on every site we deploy.

[andjsch]

I dug a bit into the problem, why URLs, that should be invalidated with the entry (listed inside config/statamic/static_caching.php) won't be invalidated.

I could see that the URLs that have been saved as a static file, aren't cached [with the cacher]. This is why they won't be invalidated, as the invalidateUrl() function exits early. I adjusted the function to look like this to make it work temporarily:

[andjsch]

Background info: I don't know why, but invalidation on this specific site has worked for a very long time with the database queue connection enabled. I think it has stopped working as soon as they started to translate some content.

I would be curious if the above fix that worked for me, works for one of you as well. Because if it does, I am happy to make a PR out of it.

[jakubjo]

Thanks @edalzell for your workaround you described in #3305!

[edalzell]

Thank @edalzell for your workaround you described in #3305!

You're welcome

[jakubjo]

I've fixed this by adding base_url to config/statamic/static_caching.php:

    /*
    |--------------------------------------------------------------------------
    | Caching Strategies
    |--------------------------------------------------------------------------
    |
    | Here you may define all of the static caching strategies for your
    | application as well as their drivers.
    |
    | Supported drivers: "application", "file"
    |
    */

    'strategies' => [
        'half' => [
            'driver' => 'application',
            'expiry' => null,
            'base_url' => config('app.url'), // <-- added this line
        ],

        'full' => [
            'driver' => 'file',
            'path' => public_path('static'),
            'lock_hold_length' => 0,
            'base_url' => config('app.url'), // <-- added this line
        ],
    ],

Everything seems fine. I'll watch the issue and report here if anything will not work as expected.

[edalzell]

I've fixed this by adding base_url to config/statamic/static_caching.php:

OH nice find, thanks!

[andrew-ireland]

adding base_url to config/statamic/static_caching.php: didn't fix my issues with static cache invalidaton, when using queues — anyone else had any luck?

Using QUEUE_CONNECTION=redis the static cache will not invalidate, even with this in place. If I use the standard QUEUE_CONNECTION=sync though, the static cache is cleared, and the invalidation rules are respected.

[dniccum]

We have experienced this item on our website as well. The work around proposed by @andrew-ireland does seem to fix the issue.