[7.x] Added queue suffix for SQS driver

[scrubmx]

When using multiple queues and environments it is hard to use different queues per environment (especially when using laravel vapor).

This PR allows suffixing the queue names by optionally adding a suffix entry to the SQS config in queue.php.

So, by adding a suffix to the config/queue.php file:

// config/queue.php

'sqs' => [
	'driver' => 'sqs',
	...
	'queue' => env('SQS_QUEUE', 'your-queue-name'),
+	'suffix' => env('SQS_SUFFIX'),
	'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
],

A job using the "imports" queue will yield distinct suffixed queue url's based on the SQS_SUFFIX environment variable, ie:

• https://sqs.someregion.amazonaws.com/account-id/imports-staging
• https://sqs.someregion.amazonaws.com/account-id/imports-production

Omitting the suffix config will yield the current (expected) behaviour:

• https://sqs.someregion.amazonaws.com/account-id/imports

Other references:

• Removed forced forward slash for SQS queue names #25610
• [5.8] Remove prefix setting for Amazon SQS, add url #27086

[GrahamCampbell]

Laravel 6.x is closed for new features now. It is accepting only bug fixes, as part of the LTS process. I will re-target this PR at the current stable: 7.x.

[taylorotwell]

This is really confusing to me. Why would you not just add your suffix onto SQS_QUEUE directly? Why would you define it separately only to have the framework eventually concatenate it anyways?

[roberto-aguilar]

I can elaborate on that using what i think would be a common scenario in Vapor.

If you have one project deployed in two environments with multiple queues, your vapor file will look like this:

environments:
    production:
        queues:
            - emails-production
            - imports-production
    staging:
        queues:
            - emails-staging
            - imports-staging

That suffix definition ensures that production queue jobs doesn't get processed on staging and vice-versa because both environments are active at the same time.

The goal in code would be to able to do this without having to think about the environments:

dispatch(new SendWelcomeEmail($user))->onQueue('emails');
dispatch(new ImportEmailsCsv($path))->onQueue('imports');

But currently that's not possible and requires the override of Illuminate\Bus\Queueable::onQueue which is not that bad, but with this PR we can avoid that and also be able to even use the $queue property right in the job definition:

class SendWelcomeEmail implements ShouldQueue
{
    /**
     * The name of the queue the job should be sent to.
     *
     * @var string|null
     */
    public $queue = 'emails';
}

All that it would take is just to define the SQS_SUFFIX environment variable in each environment and that's it! Vapor will handle this automagically 😃

What do you think, @taylorotwell? Would you be open to this with this use case in mind?

[scrubmx]

@taylorotwell

This is really confusing to me. Why would you not just add your suffix onto SQS_QUEUE directly? Why would you define it separately only to have the framework eventually concatenate it anyways?

Our application has 7 different queues and two different environments (staging and production):

• automation-production
• automation-staging
• broadcasts-production
• broadcasts-staging
• contacts-production
• contacts-staging
• default-production
• default-staging
• emails-production
• emails-staging
• imports-production
• imports-staging
• messages-production
• messages-staging

We are on laravel vapor and have to use SQS for both environments since it's cheaper for us.

As @roberto-aguilar mentioned we want to be able to define the queue name in the job class but currently is not possible. We looked into changing the prefix per environment but that does not work since we end up with an URL that looks like this:

sqs.someregion.amazonaws.com/account-id/staging-/broadcasts

Which is not correct. We also attempted to run the queues in different regions but AWS_DEFAULT_REGION is reserved in vapor and also does some magic (https://github.com/laravel/vapor-core/blob/2.0/src/ConfiguresSqs.php).

If merged this will make running multiple queues in different environments (not just in vapor but any application) easier.

Since our knowledge of AWS Lambda and Vapor, in general, is limited we are also hoping that there is an even easier approach that we are not aware of.

Thanks for your help :)

[taylorotwell]

You may want to PR laravel/laravel to add this option.