image
Blog Post • development

Asynchronous PHP with message queues

September 15, 2015by Bez Hermoso 5 min read
Blog Post • development
Asynchronous PHP with message queues
Back to top

PHP is without a doubt the most popular language used for programming server-side applications for the Web. However, despite its popularity, it is not the best-optimized of them all. In order to satisfy more modern requirements for web applications, we need to turn to other technologies to patch up some of PHP inadequacies. An example of such techologies are message queues. This blog post will give you an overview of what message queues are about, its capabilities and what type of problems they address, and how you can use them in PHP.

The Problem

PHP is like a goldfish -- it has really bad long-term memory. The traditional PHP execution model is rather short-lived: once your web server receives an HTTP request, it passes the request information to PHP which in turn loads your code into memory, executes it from head to toe, and then throws everything away upon termination of the request. If you are a PHP developer, this cycle is very familiar to you -- e.g., a variable you set in one request doesn't persist through into the next. They just go away. We use the session or a database to persist information. Whatever state you have in memory is isolated within that request and that request only, and once we return a request to the client, everything else goes down the drain, never to be seen again. For this very reason we are forced to write our code to do everything that needs to be done for that request within this window in time.

In most cases, this is OK -- but if your site visitor performs an action on your website which requires your code to do something time-consuming like sending multiple emails or calling an external API over the interwebz, the poor soul on the other side of the request has to wait until that thing is finished before he can see the resulting web-page and move along. Worse yet, if the person has gotten impatient and decided to close the web page before your task is complete, you are out of luck -- that task is now just as dead as the closed browser tab. Depending on the nature of the task that was aborted prematurely, your website data may now be in an incomplete and inconsistent state.

Compare this to the execution model of other languages and their respective web frameworks like Node.js + Express, there is a lot to be desired.

Message queues to the rescue

One way you can augment this limitation in PHP is by the use of message queues.

A message queue is a service that provides a communication protocol wherein a sender can push "messages" to a queue which a listener will receive eventually. The beauty is that the protocol is asynchronous. The action of sending is totally independent from the action of listening -- this means that you can send messages without having to wait for the listener to finish its job, let alone acknowledge that it received it. This means your application doesn't have to wait for a task to finish if doesn't have to. Pushing messaging into the queue is really fast and cheap, and message delivery is guaranteed. All it has to do is to send some form of command in the form of a message to the queue, which a listener will eventually pick up and process.

Asynchronous execution in Laravel 5.1

One notable utilization of message queues in the PHP ecosystem is Laravel 5.1's queues component. Using it allows you to break down your code execution into two parts: 1.) dispatching a job, and 2.) handling the job.

Say we have a hypothetical CMS in Laravel 5.1 in which writers can write and publish articles with the option of sharing them to various social media networks.

Let's look at the code behind publishing and sharing an article to various social networks.

We can identify the following jobs and we can task Laravel to generate them:

> php artisan make:job PublishArticle
> php artisan make:job ShareToSocialMedia --queued

These Artisan commands will generate two job classes for us. The PublishArticle job will be responsible for updating the publish state of an article in the database, while ShareToSocialMedia would be about posting the article through various social network APIs'. As you can see, we specified that the latter job as --queued -- we can identify the task of posting to communicating with multiple social network APIs can potentially be time-consuming. Specifying it as a queueable command will let Laravel know that it should be handled asynchronously via a message queue.

A lot can be said about whether or not updating an article's publish state warrants a job. I'm doing it this way to illustrate that some jobs might be light enough that they don't have to be handled through a message queue.

There are valid reasons why you want to encapsulate all business logic as jobs, though. For example, it would be a good way to structure your application if you wish to implement some form of extensive auditing & history tracking throughout your application.

Each job class generated will have a handle method which you will need to fill in with business logic that is to be executed when the job is invoked:

<?php

/* app/Jobs/PublishArticle.php */

namespace App\Jobs;

...

class PublishArticle extends Job implements SelfHandling
{
    public $article;

    public function __construct(Article $article)
    {
        $this->article = $article;
    }
    public function handle()
    {
        $this->article->update(['published' => true]);
    }
}

/* app/Jobs/ShareToSocialMedia.php */

namespace App\Jobs;

...

class ShareToSocialMedia extends Job implements SelfHandling, ShouldBeQueued
{
    public $article;
    public $socialNetworks;

    public function __construct(Article $article, array $socialNetworks)
    {
        $this->article = $article;
        $this->socialNetworks = $socialNetworks;
    }

    /* The `SocialNetworkRegistry` instance will be injected by Laravel's IoC */
    public function handle(SocialNetworkRegistry $registry)
    {
        foreach ($socialNetworks as $network) {
            $registry->get($network)->share($article);
        }
    }
}

You can then dispatch them in a controller via the dispatch method:

<?php

use App\Commands;

...

public function publish($id, Request $request)
{
    $article = Article::find($id);

    $this->dispatch(new PublishArticle($article));
    if ($request->get('networks')) {
        $this->dispatch(new ShareToSocialMedia($article, $request->get('networks'));
    }
    return redirect()->action('ArticlesController@list')->withSuccess('Article published!');
}

This published state will be updated synchronously, while the sharing to social networks will be deferred via the queue you have set-up.

To run the queue listener, execute:

> php artisan queue:listen

Have this running on your server so that jobs you dispatch are executed.

For more info about Laravel's queuing capabilities, you can read more here:

Asynchronous execution in other PHP frameworks

Laravel 5.1 seems to be the only highly-popular framework that has built-in support for asynchronous execution via message queues that is excellent. It is so easy to get into. If you are using other frameworks, you are still covered (to some extent).

Symfony2, my framework of choice, doesn't have built-in support for message queues in its standard edition. However there are a couple of community bundles that allows you to integrate message queues into your Symfony application more conveniently. You can read more in this blog post.

Zend has Zend\Queue. However it is not as mature as Laravel's implementation.

However, both of these doesn't have Laravel's Jobs and Events abstraction on top of it.

All in all, I must say Laravel's integration with message queues and the abstractions on top of it is top-notch. Given how strongly Laravel developers feel about following "The Laravel Way", I think this can only benefit the PHP community by having a growing community gaining wisdom in using asynchronous execution and evangelizing it to improve web app performance.

We at ActiveLAMP had the pleasure of using RabbitMQ and IronMQ for our message queue needs. RabbitMQ is an open-source, free-to-use, message queue that you can install on your own servers and manage yourself. IronMQ is a cloud-based alternative. Both of them have PHP SDKs you can use: You can utilize videlalvaro/php-amqplib to interact with RabbitMQ and IronMQ has its own library published as iron-io/iron_mq_php. You can use these services to implement asynchronous execution in your application if it doesn't natively support it. They are really easy to integrate.

You should do your own research, assess your needs, and choose accordingly among the two services and many others out there.

Authored by