Unrecognized UTF8 Byte

[hipsterelitist]

Hey,

Some of our requests seem to be having UTF8 encoding issues when using the new API. Running a request with the same html body (I'm assuming that's where this fall) against the old API results in a successful send.

From what I can tell, it is rather similar to this issue brought up in the Go client: sendgrid/sendgrid-go#66

Error below.

{"errors":[{"message":"Unrecognized UTF8 Byte at position 31051","field":null,"help":"http://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/errors.html#-Encoding-Errors"}]}

Can provide more details upon request, but just wanted to bring it to your attention since it seems to be impacting multiple clients.

Technical details:

• sendgrid-ruby Version: master (latest commit: [commit number])
• Ruby Version: 2.2.3

[thinkingserious]

@hipsterelitist ,

This bug is currently being fixed and I have added you to the list of people to be notified when it's done. Thanks!

[pbyrne]

We're experiencing the same error. Could you add me to the list of notified people?

In case this helps: in attempt to narrow down the root cause, we've found that we can't reproduce the error when only using sendgrid-ruby in a sample app. After loading active_support/json (which provides a new #to_json method), we can consistently reproduce with certain payloads. I suspect that the trouble seems to be that ActiveSupport's JSON encoding escapes HTML characters like < and >, while Ruby's default JSON encoding doesn't.

Best I can tell, both forms are valid:

# Just plain Ruby
["<foo>"].to_json
# => "[\"<foo>\"]"
JSON.parse(["<foo>"].to_json)
# => ["<foo>"]

# After `require 'active_support/json'`
["<foo>"].to_json
# => "[\"\\u003cfoo\\u003e\"]"
JSON.parse(["<foo>"].to_json)
# => ["<foo>"]

[thinkingserious]

You have been added @pbyrne and thanks for the additional feedback!

[ming535]

@thinkingserious is there a version that don't have this issue for v3 api?

[thinkingserious]

@ming-relax,

Unfortunately not. This is a server side issue.

That said, as of right now, it looks like this fix is scheduled to be released today.

[thinkingserious]

The fix for this has been deployed :)

Thank you for your patience!