SendGrid not relaying emails from Klaxon when using the Heroku setup

[harrislapiroff]

I was trying to get this set up on Heroku using the one-click process. I had to go through the SendGrid rigamarole listed in the README, but even then I wasn't receiving the confirmation email when I attempted to log in. I dug into the logs and found this entry:

[ActiveJob] [ActionMailer::DeliveryJob] [ef565f89-dc0b-4535-9904-f123e268ba3f] Error performing ActionMailer::DeliveryJob (Job ID: ef565f89-dc0b-4535-9904-f123e268ba3f) from Async(mailers) in 577.69ms: Net::SMTPFatalError (550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developers/sending-email/sender-identity/ to see the Sender Identity requirements

It appears that Klaxon is sending the confirmation email from the address Klaxon <[email protected]> and SendGrid is rejecting it since it's not a verified sender on my account. I'm not sure what the general solution is here, but here's how I resolved it:

1. Went through SendGrid support process to get my account unsuspended, as specified in the readme

2. From the SendGrid dashboard, created a verified sender using my personal email address. Clicked the confirmation link from my email when received.

3. From the SendGrid dashboard, provisioned an API key for Klaxon.

4. From Heroku, updated env vars as follows:

   SENDGRID_USERNAME = apikey
SENDGRID_PASSWORD = API key from SendGrid
   MAILER_FROM_ADDRESS = My personal email address

I'm not sure that provisioning the API key and updating the username/password was a necessary part of this process, but it seemed like a wise thing to do.

[tommeagher]

@harrislapiroff thank you for filing this excellent issue. This is super helpful. Sendgrid has been getting progressively more and more difficult to deal with and has changed much of the workflow since we launched Klaxon 4 years ago. We have to think through how to sort this out to make it easier for everyone to get started. This will be indispensable in doing that. Thanks!

[tszumowski]

@tommeagher I also ran into this issue. Can I suggest that the README be updated with this SendGrid situation and the MAILER_FROM_ADDRESS to be set in the way @harrislapiroff described? Coming in fresh last night, it took me quite a while to discover the issue. Particularly because the README implies it's an issue on SendGrid's side requiring a support ticket.

[GabeIsman]

Good suggestion @tszumowski, I've added some notes on this to the doc. Also thank you @harrislapiroff (also hi! funny seeing you here) for detailing this issue and the fix so clearly.

[tszumowski]

@GabeIsman @harrislapiroff I noticed when I use my personal email (the one used to verify sender), it hits the SendGrid limit of 100 emails a day, whereas Heroku claims up to 12000 per month, as does the README here.

I ended up quickly hitting that 100 emails a day limit due to using my personal email (for reasons described above). Any way to get that 12000 emails a month via Heroku?

[tommeagher]

@tszumowski this is a new wrinkle we haven't heard about yet. Is the email you used to verify with Sendgrid the same email you use for your Heroku account?
@harrislapiroff have you run into this?

[tszumowski]

@tommeagher

Is the email you used to verify with Sendgrid the same email you use for your Heroku account?

Yep. Both are my personal gmail account.

I opened a support ticket with SendGrid. They said my account wasn't suspended or anything. And they said to work with Heroku if I am running into limitations with the Heroku app.

[tommeagher]

Good to know, though I am stumped. @GabeIsman have you heard of this issue coming up?

[GabeIsman]

Hmm, seems like yet another instance of Sendgrid not really supporting their Heroku plugin. That was the same limitation I ran into when investigating the free tier of mailgun. There just doesn't seem to be a proper free solution for sending emails from Heroku anymore. I'm not sure what to tell you @tszumowski. You may need to 1) pay for Sendgrid or 2) replace Sendgrid with something like Postmark that has a (likely cheaper) pay-as-you-go model. I wish there were an easier and freer way to do this on Heroku!

[tszumowski]

@GabeIsman no worries! And thanks for the ideas! I ended up making good use of the slack notification which ends up fitting my use cases better anyway! For SendGrid, 100/day is still a good backup for me for personal use, and I imagine for people to get started with Klaxon. I just happened to have some real chatty sites during my initial testing. Thank you again!

[tommeagher]

@tszumowski I'm sorry about that. So weird that the support from Sendgrid has deteriorated so much in the last year. Let us know if Heroku has any answer or advice.

[immewnity]

(late reply, but figured this is useful info)
Looks like SendGrid changed how their free tier works (per https://www.reddit.com/r/webdev/comments/6flnhz/free_sendgrid_account_is_limited_to_100_emails/), but if you had an account set up before then, you're grandfathered in to the 12,000. It also looks like you need to sign up for a trial to get the 100/day, else it defaults to 100/month, though that doesn't seem to be happening here.

Several of the alternatives mentioned in the Reddit comments have also since changed their free tiers, so... there may not be a better option.

[immewnity]

Setting up a new instance, this is what SendGrid is now asking when you request the account to be unlocked:

[tommeagher]

Geez. What a pain. We're considering now alternatives to SendGrid for new installations. This path doesn't seem tenable any longer for new users.

[immewnity]

Forgot to update that SendGrid denied my account from being activated, sooooo yeah. Another method would be good 😂

[immewnity]

Not that more reasons to move away from SendGrid are really needed, but Microsoft now blocks several SendGrid free IPs from sending email to Outlook/Live/Hotmail accounts. Here's a reply I received while troubleshooting (this isn't the only IP I've seen blocked):

We have investigated your deliverability issue regarding IP: (167.89.10.203 ). At the current time, these IPs are not eligible for mitigation.

We are concerned with a pattern of irregular mail traffic originating from your IPs. While this could be benign, sudden changes in email sending volume can be the result of a server or network compromise or the result of compromised user accounts.

[rdmurphy]

I don't see it in Heroku's add-on's list so you'd have to create your own account outside of Heroku's ecosystem, but I'm curious if anyone has ever tried SMTP2GO. Has a reasonably decent free plan but I fear it may suffer from some of the same issues unless you have a domain to set up with it, though.

[immewnity]

Got a working Mailgun integration at /immewnity/klaxon/tree/mailgun - this branch also includes the Microsoft Teams integration proposed in #514 and the develop fix in #578 .

[tommeagher]

Hi all. With the recent release of Klaxon Cloud, we're going back and revisiting old issues that we're not going to pursue or support as we consider any future development of the original standalone Klaxon. This one (almost 4 years old now) falls in that bucket. Thanks for the contributions and discussions on this, but we'll close it as WONTFIX.