Strange characters in email and database

[John Langer]

Hi,
Not sure which forum to use for this. I'm guessing it's iRite but it could be a UE or even a DA issue.

I've constructed an emailing to mailing list system using these three products (DA, UE and iRite). I used DA to to create the admin area with a submit and update form. I have a text area in the form to submit the body text of the email. In this area I inserted iRite so that the email body can be formated. On the details page of DA I've inserted the UE behaviour to send the email.

The client has just sent out an email and I was one of the recipients. What he wrote was "Harvest Offering" but the email says ‘Harvest Offering’. I've looked in the MySql database and what is inserted there is ‘< b >Harvest Offering< /b >’ (I've added the spaces in < b > to prevent rendering on this forum).

Any help would be appreciated.

[Eric Mittman]

Check the initial value that is being used for the iRite editor. It looks like you may be doing some encoding on the value and getting the strange characters as a result. This is my best guess based on what you have here and that the value is also stored in the db this way.

When the message is sent is the body being set to the value of the iRite instance, or are you storing the value in your db first, then recalling it with a recordset to add it into the email body?

[John Langer]

Hi Eric, thanks for the reply.
Check the initial value that is being used for the iRite editor. It looks like you may be doing some encoding on the value and getting the strange characters as a result. This is my best guess based on what you have here and that the value is also stored in the db this way.
Not sure what you mean by the initial value to be honest. It's just a form with iRite added.

When the message is sent is the body being set to the value of the iRite instance, or are you storing the value in your db first, then recalling it with a recordset to add it into the email body?
The content is submitted into the database and then retrieved from there for the email.

[Eric Mittman]

What I was referring to for the initial value is for the iRite editor. If there is some formatting in the initial value it can cause issues with the characters. You can check this by editing the iRite instance and looking at the initial value on the first screen.

As for the db part since you are inserting the record into your db first it could potentially be an issue with that part as well. To determine if the characters are being inserted into the db like this you should print out the insert to see if the characters are present.

To print out the insert you can just find the WA_SQL variable in the code, it looks like this in PHP:


$WA_Sql


On the next line after this variable you can print out the query with a die statement like this:


die($WA_Sql);


Please give this a try to determine at one point the bad characters are being added and post back with the result you get.

[John Langer]

Hi Eric,

Now I've had a bit of time to really look at this I'm thinking that there isn't really a problem. I've sent numerous test messages and just cannot replicate the issue. I'm guessing that perhaps the client pasted directly from "Word" perhaps and that "did it". I just don't know. I have asked them but as yet had no reply.

Answering your question, the initial value was just ordinary text "Write you message here" (without the inverted commas).

If it turns out to be a real problem then I'll post back.

Thanks for your time and sorry to have wasted it.

[Eric Mittman]

Definitely let us know if the issues persists. Sometimes the best thing that can happen is for the problem to not happen again, but it is never very satisfying not knowing what was happening.

[Blikbrein]

I basically have the same setup as the person above. DA used to insert and update records with html editor in the fields for client to insert content which is then mailed as a newsletter to their subscribers.

Here is how a paragraph should look and does look when viewing in Apple Mail:

amazingly… Patrick Liew form Lancôme arranged that we spend a day at The Lancôme Institute, and if that was not enough, ended with getting our hair treated at the L’Oreal Hair Spa!

Here is the same paragraph viewed in Outlook or outlook express:

amazingly… Patrick Liew form Lancôme arranged that we spend a day at The Lancôme Institute, and if that was not enough, ended with getting our hair treated at the L’Oreal Hair Spa!

I thought it could be content pasted from Word that contained some sort of formatting, but tested the content by typing and not copying from Word. Got the same results as above.

Then I thought maybe the default content contained in the field when inserting a new record. It consisted of 3 pre-formatted tables. Also deleted it in the html editor settings and then typed text. Did not change the issue.

When viewing the content on the web, it also looks fine. It is only when mailed to recipients with Windows mail clients.

Any ideas?

[Eric Mittman]

To ensure that there is no extra formatting occurring you should take a look at the data that is in the db. If there is any extra formatting here you should be able to see it.

If there is no extra characters or formatting present in your db or on the pages in your site then it would indicate that the problem is how Outlook is rendering the characters in the message.

You can also test this out by sending a message to a web based email account to see if the characters show in there.

Please report back and let us know what you discover, if it is only outlook that is rendering the characters differently then you may need to adjust the encoding that Outlook is using.

[Blikbrein]

I checked how the text renders in gmail and it looks fine. So I went to the specific field in the db and saw that the characters are incorrect in the db already:

amazingly… Patrick Liew form Lancôme arranged that we spend a day at The Lancôme Institute, and if that was not enough, ended with getting our hair treated at the L’Oreal Hair Spa!</p> <p>

I then remembered the same issue I had about 6 months ago with the database encoding on another website. An experienced web developer helped me to correct things, but I cannot remember where to change the code. He gave me this line to replace in that instance:

Replace sql_set_charset:

mysql_query("SET DBFIELD 'utf8'", $mydbname) or die(mysql_error());

I know the problem is caused by the db encoding. How can I correct this?

[Blikbrein]

Changed the mail encoding to utf-8 in Universal e-mail general tab and mails are displaying correctly in outlook.

[Eric Mittman]

That is great to hear, I know that adjusting the charset can help with character rendering under some conditions but there are also times when the mail client itself will use a predetermined character set for the encoding. It is best to make sure that you have it matching along the way to avoid this if it is an option for you.

I'm sure others will benefit from the post, thanks for getting back with that.