Laravel — Storing Failed Jobs In A Json File

Kevin Bui — Mon, 04 Nov 2024 13:34:02 GMT

Laravel — Storing Failed Jobs In A Json File

Failed queued jobs in Laravel are typically stored in the failed_jobs table.

Being a big fan of Laravel, I regularly read the source code, picking up secret features and occasionally contribute.

Lately, I have found out that there is an alternative to storing failed jobs in Laravel. which is a JSON file!

1. Configurations:

In the config/queue.php, this is the default configuration:

[
    //...
    'failed' => [
        'driver' => env('QUEUE_FAILED_DRIVER', 'database-uuids'),
        'database' => env('DB_CONNECTION', 'sqlite'),
        'table' => 'failed_jobs',
    ],
];

In the .env file, we will set the QUEUE_FAILED_DRIVER variable to be file:

QUEUE_FAILED_DRIVER=file

That’s all we have to do! Now, lets test storing the failed jobs in a JSON file.

2. Test Storing Failed Jobs To JSON Files:

Now, I define a minimal, simple job that will always fail.


namespace App\Jobs;

use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ThisJobWillFail implements ShouldQueue
{
    use Queueable;

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        $this->fail('This job will always fail.');
    }
}

I dispatch this job 3 times:

use App\Jobs\ThisJobWillFail;
use Queue;

ThisJobWillFail::dispatch();
ThisJobWillFail::dispatch();
ThisJobWillFail::dispatch();

// Or
Queue::bulk([
    new ThisJobWillFail(),
    new ThisJobWillFail(),
    new ThisJobWillFail(),
]);

Then, obviously we need to run the queue worker:

php artisan queue:work

   INFO  Processing jobs from the [default] queue.

  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... RUNNING
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 5.30ms FAIL
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 11.90ms DONE
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... RUNNING
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 1.74ms FAIL
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 3.62ms DONE
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... RUNNING
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 1.59ms FAIL
  2024-11-04 12:32:17 App\Jobs\ThisJobWillFail .......... 3.26ms DONE

Now, opening the storage/framework/cache/failed-jobs.json, we have all the jobs the previously failed!

[
    {
        "id": "3bbd74b2-dbfe-4b95-b7f6-b6be807ed8bf",
        "connection": "database",
        "queue": "default",
        "payload": "{\"uuid\":\"3bbd74b2-dbfe-4b95-b7f6-b6be807ed8bf\",\"displayName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"job\":\"Illuminate\\\\Queue\\\\CallQueuedHandler@call\",\"maxTries\":null,\"maxExceptions\":null,\"failOnTimeout\":false,\"backoff\":null,\"timeout\":null,\"retryUntil\":null,\"data\":{\"commandName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"command\":\"O:24:\\\"App\\\\Jobs\\\\ThisJobWillFail\\\":0:{}\"}}",
        "exception": "Illuminate\\Queue\\ManuallyFailedException: This job will always fail. in \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/InteractsWithQueue.php:56\nStack trace:\n#0 \/Users\/kevinbui\/Code\/test-laravel-11\/app\/Jobs\/ThisJobWillFail.php(17): App\\Jobs\\ThisJobWillFail->fail('This job will a...')\n#1 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): App\\Jobs\\ThisJobWillFail->handle()\n#2 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#3 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#4 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#5 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#6 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(128): Illuminate\\Container\\Container->call(Array)\n#7 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Bus\\Dispatcher->Illuminate\\Bus\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#8 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#9 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(132): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#10 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(124): Illuminate\\Bus\\Dispatcher->dispatchNow(Object(App\\Jobs\\ThisJobWillFail), false)\n#11 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Queue\\CallQueuedHandler->Illuminate\\Queue\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#12 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#13 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(123): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#14 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(71): Illuminate\\Queue\\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(App\\Jobs\\ThisJobWillFail))\n#15 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Jobs\/Job.php(102): Illuminate\\Queue\\CallQueuedHandler->call(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Array)\n#16 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(440): Illuminate\\Queue\\Jobs\\Job->fire()\n#17 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(390): Illuminate\\Queue\\Worker->process('database', Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(Illuminate\\Queue\\WorkerOptions))\n#18 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(177): Illuminate\\Queue\\Worker->runJob(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), 'database', Object(Illuminate\\Queue\\WorkerOptions))\n#19 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(148): Illuminate\\Queue\\Worker->daemon('database', 'default', Object(Illuminate\\Queue\\WorkerOptions))\n#20 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(131): Illuminate\\Queue\\Console\\WorkCommand->runWorker('database', 'default')\n#21 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()\n#22 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#23 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#24 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#25 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#26 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(213): Illuminate\\Container\\Container->call(Array)\n#27 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Command\/Command.php(279): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#28 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(182): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#29 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(1047): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#30 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(316): Symfony\\Component\\Console\\Application->doRunCommand(Object(Illuminate\\Queue\\Console\\WorkCommand), Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#31 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(167): Symfony\\Component\\Console\\Application->doRun(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#32 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Console\/Kernel.php(197): Symfony\\Component\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#33 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Application.php(1203): Illuminate\\Foundation\\Console\\Kernel->handle(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#34 \/Users\/kevinbui\/Code\/test-laravel-11\/artisan(13): Illuminate\\Foundation\\Application->handleCommand(Object(Symfony\\Component\\Console\\Input\\ArgvInput))\n#35 {main}",
        "failed_at": "2024-11-04 12:32:17",
        "failed_at_timestamp": 1730723537
    },
    {
        "id": "42f84bdb-c1fd-4a1b-93c6-24babcf702b3",
        "connection": "database",
        "queue": "default",
        "payload": "{\"uuid\":\"42f84bdb-c1fd-4a1b-93c6-24babcf702b3\",\"displayName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"job\":\"Illuminate\\\\Queue\\\\CallQueuedHandler@call\",\"maxTries\":null,\"maxExceptions\":null,\"failOnTimeout\":false,\"backoff\":null,\"timeout\":null,\"retryUntil\":null,\"data\":{\"commandName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"command\":\"O:24:\\\"App\\\\Jobs\\\\ThisJobWillFail\\\":0:{}\"}}",
        "exception": "Illuminate\\Queue\\ManuallyFailedException: This job will always fail. in \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/InteractsWithQueue.php:56\nStack trace:\n#0 \/Users\/kevinbui\/Code\/test-laravel-11\/app\/Jobs\/ThisJobWillFail.php(17): App\\Jobs\\ThisJobWillFail->fail('This job will a...')\n#1 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): App\\Jobs\\ThisJobWillFail->handle()\n#2 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#3 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#4 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#5 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#6 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(128): Illuminate\\Container\\Container->call(Array)\n#7 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Bus\\Dispatcher->Illuminate\\Bus\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#8 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#9 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(132): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#10 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(124): Illuminate\\Bus\\Dispatcher->dispatchNow(Object(App\\Jobs\\ThisJobWillFail), false)\n#11 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Queue\\CallQueuedHandler->Illuminate\\Queue\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#12 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#13 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(123): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#14 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(71): Illuminate\\Queue\\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(App\\Jobs\\ThisJobWillFail))\n#15 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Jobs\/Job.php(102): Illuminate\\Queue\\CallQueuedHandler->call(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Array)\n#16 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(440): Illuminate\\Queue\\Jobs\\Job->fire()\n#17 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(390): Illuminate\\Queue\\Worker->process('database', Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(Illuminate\\Queue\\WorkerOptions))\n#18 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(177): Illuminate\\Queue\\Worker->runJob(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), 'database', Object(Illuminate\\Queue\\WorkerOptions))\n#19 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(148): Illuminate\\Queue\\Worker->daemon('database', 'default', Object(Illuminate\\Queue\\WorkerOptions))\n#20 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(131): Illuminate\\Queue\\Console\\WorkCommand->runWorker('database', 'default')\n#21 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()\n#22 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#23 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#24 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#25 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#26 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(213): Illuminate\\Container\\Container->call(Array)\n#27 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Command\/Command.php(279): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#28 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(182): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#29 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(1047): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#30 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(316): Symfony\\Component\\Console\\Application->doRunCommand(Object(Illuminate\\Queue\\Console\\WorkCommand), Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#31 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(167): Symfony\\Component\\Console\\Application->doRun(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#32 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Console\/Kernel.php(197): Symfony\\Component\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#33 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Application.php(1203): Illuminate\\Foundation\\Console\\Kernel->handle(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#34 \/Users\/kevinbui\/Code\/test-laravel-11\/artisan(13): Illuminate\\Foundation\\Application->handleCommand(Object(Symfony\\Component\\Console\\Input\\ArgvInput))\n#35 {main}",
        "failed_at": "2024-11-04 12:32:17",
        "failed_at_timestamp": 1730723537
    },
    {
        "id": "596f3431-335e-49de-9721-0b7db92fa580",
        "connection": "database",
        "queue": "default",
        "payload": "{\"uuid\":\"596f3431-335e-49de-9721-0b7db92fa580\",\"displayName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"job\":\"Illuminate\\\\Queue\\\\CallQueuedHandler@call\",\"maxTries\":null,\"maxExceptions\":null,\"failOnTimeout\":false,\"backoff\":null,\"timeout\":null,\"retryUntil\":null,\"data\":{\"commandName\":\"App\\\\Jobs\\\\ThisJobWillFail\",\"command\":\"O:24:\\\"App\\\\Jobs\\\\ThisJobWillFail\\\":0:{}\"}}",
        "exception": "Illuminate\\Queue\\ManuallyFailedException: This job will always fail. in \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/InteractsWithQueue.php:56\nStack trace:\n#0 \/Users\/kevinbui\/Code\/test-laravel-11\/app\/Jobs\/ThisJobWillFail.php(17): App\\Jobs\\ThisJobWillFail->fail('This job will a...')\n#1 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): App\\Jobs\\ThisJobWillFail->handle()\n#2 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#3 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#4 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#5 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#6 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(128): Illuminate\\Container\\Container->call(Array)\n#7 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Bus\\Dispatcher->Illuminate\\Bus\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#8 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#9 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Bus\/Dispatcher.php(132): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#10 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(124): Illuminate\\Bus\\Dispatcher->dispatchNow(Object(App\\Jobs\\ThisJobWillFail), false)\n#11 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(144): Illuminate\\Queue\\CallQueuedHandler->Illuminate\\Queue\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#12 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(App\\Jobs\\ThisJobWillFail))\n#13 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(123): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#14 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/CallQueuedHandler.php(71): Illuminate\\Queue\\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(App\\Jobs\\ThisJobWillFail))\n#15 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Jobs\/Job.php(102): Illuminate\\Queue\\CallQueuedHandler->call(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Array)\n#16 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(440): Illuminate\\Queue\\Jobs\\Job->fire()\n#17 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(390): Illuminate\\Queue\\Worker->process('database', Object(Illuminate\\Queue\\Jobs\\DatabaseJob), Object(Illuminate\\Queue\\WorkerOptions))\n#18 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Worker.php(177): Illuminate\\Queue\\Worker->runJob(Object(Illuminate\\Queue\\Jobs\\DatabaseJob), 'database', Object(Illuminate\\Queue\\WorkerOptions))\n#19 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(148): Illuminate\\Queue\\Worker->daemon('database', 'default', Object(Illuminate\\Queue\\WorkerOptions))\n#20 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Queue\/Console\/WorkCommand.php(131): Illuminate\\Queue\\Console\\WorkCommand->runWorker('database', 'default')\n#21 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()\n#22 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Util.php(43): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#23 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(95): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))\n#24 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/BoundMethod.php(35): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))\n#25 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Container\/Container.php(690): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)\n#26 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(213): Illuminate\\Container\\Container->call(Array)\n#27 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Command\/Command.php(279): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#28 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Console\/Command.php(182): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))\n#29 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(1047): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#30 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(316): Symfony\\Component\\Console\\Application->doRunCommand(Object(Illuminate\\Queue\\Console\\WorkCommand), Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#31 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/symfony\/console\/Application.php(167): Symfony\\Component\\Console\\Application->doRun(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#32 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Console\/Kernel.php(197): Symfony\\Component\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#33 \/Users\/kevinbui\/Code\/test-laravel-11\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Application.php(1203): Illuminate\\Foundation\\Console\\Kernel->handle(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))\n#34 \/Users\/kevinbui\/Code\/test-laravel-11\/artisan(13): Illuminate\\Foundation\\Application->handleCommand(Object(Symfony\\Component\\Console\\Input\\ArgvInput))\n#35 {main}",
        "failed_at": "2024-11-04 12:32:17",
        "failed_at_timestamp": 1730723537
    }
]

3. How Does It Work?

In the Illuminate\Queue\QueueServiceProvider class, lets have a look at the registerFailedJobServices method:

if (isset($config['driver']) && $config['driver'] === 'file') {
    return new FileFailedJobProvider(
        $config['path'] ?? $this->app->storagePath('framework/cache/failed-jobs.json'),
        $config['limit'] ?? 100,
        fn () => $app['cache']->store('file'),
    );
} elseif (isset($config['driver']) && $config['driver'] === 'dynamodb') {
    return $this->dynamoFailedJobProvider($config);
} elseif (isset($config['driver']) && $config['driver'] === 'database-uuids') {
    return $this->databaseUuidFailedJobProvider($config);
} elseif (isset($config['table'])) {
    return $this->databaseFailedJobProvider($config);
} else {
    return new NullFailedJobProvider;
}

We notice four storing mechanism here.

file (which is surprisingly the first check!)
dynamodb
database-uuids (using the uuid field to identify jobs)
database (using the id field to identify jobs)

4. Additional Configurations:

Reading the above code snippet, we easily recognise two more optional configurations for storing failed jobs with a JSON file:

[
    //...
    'failed' => [
        'driver' => env('QUEUE_FAILED_DRIVER', 'file'),
        'path' => env('QUEUE_FAILED_PATH', 'framework/cache/my-failed-jobs.json'),
        'limit' => env('QUEUE_FAILED_LIMIT', 1000),
    ],
];

5. Viewing Failed Jobs:

Similar to any other storing mechanism, we can list all failed jobs from the json file with the php artisan queue:failed command:

php artisan queue:failed

2024-11-04 12:32:17 3bbd74b2-dbfe-4b95-b7f6-b6be807ed8bf .......... database@default App\Jobs\ThisJobWillFail
2024-11-04 12:32:17 42f84bdb-c1fd-4a1b-93c6-24babcf702b3 .......... database@default App\Jobs\ThisJobWillFail
2024-11-04 12:32:17 596f3431-335e-49de-9721-0b7db92fa580 .......... database@default App\Jobs\ThisJobWillFail

6. Finally, A Few Thoughts:

I believe this mechanism would be extremely helpful for projects utilising NoSql or Graph databases. Currently I cannot think of a different approach to store failed jobs for those.

Stories by Kevin Bui on Medium

Laravel — Storing Failed Jobs In A Json File