Blog Details
- technology
- October 10, 2022
Laravel Worker with super-visor
Many of us struggled to set up a supervisor with Laravel Worker in our docker container or web server. If you don’t then you are lucky not like me. 😉
If you are struggling to do or you want to know about what is a job, how a supervisor maintains the jobs inside a docker-container or web server, and how to set up all these stuff then you are in the right post I guess.
Follow my journey that let you know something informative.
After migrating, You should find 2 tables named failed_jobs & jobs. If you don’t then follow the procedure again or check laravel queue for better understanding.
You should see table named job_batches. To learn more about batch jobs, Check the beautiful documentation of batch jobs laravel
What do we do here?
Ok, we’ve imported a CSV with 100 thousand users’ information ex. email, username, address, etc. TestCsvForSupervisor is the job class for handling some checks or operations like inserting, updating, or comparing data with any table ex. users. Simple as writing an artisan command. 😉
In TestCsvForSupervisor Job Class:
Tough one, I have 20 thousand users’ information in my user’s table, and what I’ve done here? Compare CSV data with users’ table data and store the checking result in the log (You might find it clueless but it’s just a test case actually not a good test case😒). Then dispatch the job to store the pending jobs to be done in the table.
Install from Dockerfile using Daemon
In your app/docker-conf/php/Dockerfile, Write the following script:
Then, build and start the docker:
I will suggest you follow the docker daemon-based (step-2) supervisor because it can be used globally. So, we will be continuing with that.
In your root directory make a file named laravel-worker.conf and write and save the file:
In the docker bash, you will find a file named supervisord.conf in /etc/supervisor/ directory. Overwrite the last line of the supervisord.conf file by following the script and save it:
Congrats 🥳, You’ve just done a great job not just laravel-job but the real automated jobs using supervisor. You may find the job list with status and unique id from the job_batches table after starting supervisor below shot:
There is one interesting fact about supervisors. Do you know what language is used to develop a supervisor?
It’s Python 🐍😲
Pre-requisites
- Setup laravel queue:
php artisan queue:table
php artisan migrate
- Setup job batches to keep track of jobs:
php artisan queue:batches-table
php artisan migrate
- Setup some test methods for batch jobs:
public function testJob() {
$path = storage_path('test');
$files = glob("$path/user.csv"); // Parse the stored csv
$header = [];
foreach ($files as $key => $file) {
$data = array_map('str_getcsv', file($file));
if($key === 0) {
$header = $data[0];
unset($data[0]);
}
}
$batch = Bus::batch([new TestCsvForSupervisor($header,
$data)->dispatch();
return $batch->id;
}
public function handle()
{
try{
foreach ($this->data as $row) {
$result = array();
$result = User::whereName($row[0])->first() ? 'Found' : 'Not Found';
Log::info('User checked successfully with email: '.$row[1]);
}
} catch (Exception $err) {
Log::info('Error occured with message: '.$err);
}
}
Setup supervisor
Now it’s time for setup the supervisor into the docker container where you can follow 2 steps:- You can access docker bash and install supervisor by command scripts (It is for local testing not for global deployment or team collaboration)
- You can integrate the command scripts in Dockerfile.
Supervisor is basically linux based daemon that runs in the background. You have to configure your laravel app so that supervisor know about background jobs. It monitors the executing or executable jobs and if for any interference the jobs accidentally stop ex. server down after the restarting the supervisor automatically starts the job from where it stops and monitor the failed, pending and success jobs through this time and can deliver us jobs report.Install from docker bash:
sudo apt-get install supervisor
RUN apt-get install -y supervisor
docker compose build
docker compose up -d // -d for running the docker in background
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /app/artisan queue:work // you may use extra attributes like --sleep=3
autostart=true
autorestart=true
# stopasgroup=true
# killasgroup=true
;user=
numprocs=8
redirect_stderr=true
stdout_logfile=/app/storage/logs/worker.log
stopwaitsecs = 100
[include]
files = /etc/supervisor/conf.d/*.conf /app/laravel-worker.conf
After that, Execute the following commands to initiate the supervisor:
# Start the service
service supervisor start
# Read the new config from supervisord file
supervisorctl reread
# Activate the configuration
supervisorctl update
# Start queue command for running jobs
supervisorctl start laravel-worker:* or laravel-worker
# Check the status of new config operation
supervisorctl status
Congrats 🥳, You’ve just done a great job not just laravel-job but the real automated jobs using supervisor. You may find the job list with status and unique id from the job_batches table after starting supervisor below shot:
There is one interesting fact about supervisors. Do you know what language is used to develop a supervisor?
It’s Python 🐍😲
Thanks for reading with so much patience. ❤️
nnhadmin