Setting up a catch-all email forwarder and parsing emails using…

In one of my ongoing projects i had to set up a “catch-all” email parsing script. Here’s how it came out to be.

What is catch-all?

In the context of emails, “catch-all” refers to a destination to which all incoming emails with in-correct email addresses are delivered or forwarded. As a destination, you can set either an email or a script on your server which captures such emails and save them in to your local database or files.

Most of the web servers do not turn the “catch-all” on by default as spammers may abuse this service badly. For example, if you set youremail@yourdomain.com as a catch-all email forwarding destination,  emails having email address such as abcdefgh@yourcomain.com or GJ9089khkjksdghfsd@yourdomain.com is surly going to be dropped into your inbox.

What i needed in practice

Here’s the situation. Any visitor would create a subscriber’s account on my website and thereafter would add some user profiles(contacts). Each added profile would include one or more emails addresses to which the subscriber would send periodic update requests or new updates. The subscriber would expect all his/her contacts to respond to these requests/updates in timely manner. The contacts of the subscriber are supposed to reply from their email inboxes and the responses sent back by them would be logged into my server’s database and show up on my subscriber’s account page, as an ongoing conversation under each contact name.

How it was achieved

First of all i set up a “catch-all” email forwarder targetting a php script placed on my server (i had to make a special request to my hosting provider to turn “catch-all” on for me). Fyi, i was on cpanel11 and only thing i had to do was to place http://www.mydomain.com/emailcatch.php script url into a text field and hit “Save”. The first step was done.

Next thing i needed was a good email parser to parse incoming emails. I decided to write my own script and i succeeded to some extent but soon i realized that the different email clients or service providers like yahoo!, gmail, hotmail,  thunderbird etc. had a slighly varying (not always though only sometimes) email format in terms of Header or Content or both. I needed to add more filters everytime i discovered a difference in formatting and it seemed to be endless process until i found a very thoroughly written mimeparser php class to parse emails written by Manuel Lemos. It came out to be exactly what i was really looking for.

Sending emails to my contacts

Here’s the process. I am a subscriber and i have two contacts added. One is Suresh Krishan with two email addresses i.e. suresh.Kr987ish@gmail.com and dev.surkr@yahoo.com. Another is David Samuel having email address davidSkratisjer@chimps.com. I visit my account page, do select these two contacts, writes some message in the text box and do click “Send”. In the back-end i create a random and unique sender email address, something like <UID34AUTH8976875757@mydomain.com>which includes my account id (UID:34) and unique identifier (8976875757) to identify the replies. In this case i create two email addresses 1)  <UID34AUTH8976875757@mydomain.com> as a sender for Suresh Krishan and <UID34AUTH002345758@mydomain.com> for David Samuel.

Receiving emails from both users

Suresh Krishan opens the email in his inbox and replies to <UID34AUTH8976875757@mydomain.com>. As this is a non-existing email address the “catch-all” throws this email to my script http://www.mydomain.com/emailcatch.php. In the scrip i capture the content of the email and do parse it with the help of mimeparser. My account id and unique identifiers are matched and the messages is inserted to the database at its proper place for further use.

About email parsing script

The script emailcatch.php (permissions set to 0755) should start with any one of the following “hasbangs” at the top following withe the php code to do the parsing or inserting etc:

#!/usr/local/bin/php -q
#!/usr/bin/php -q

<?php
$mimeparser_path = '/home/mydomain.com/public_html/mimeparser'; //path to directory where mimeparser script is
require_once($mimeparser_path . '/rfc822_addresses.php');
require_once($mimeparser_path . '/mime_parser.php');

/* Read the message from STDIN */
$fd = fopen("php://stdin", "rb");
$data = ""; // This will be the variable holding the data.
while (!feof($fd)) {
$data .= fread($fd, 1024);
}
fclose($fd);

............
............
More code to process the data furthe...........
?>

In the above php code $data is the data string which is supposed to be passed to mime parser class methods for parsing. If you are really interesting in looking at the full working script download it here.

Summary:

To make catch-all parse your emails to the database these are the summarised steps:

  1. Setup catch-all at server control panel (cpanel, WHM or whatever). Ask your hosting provider to help you.
  2. Create php script to catch and parse emails. Set hasbang, #!/usr/local/bin/php -q OR #!/usr/bin/php -q at the top of this file and outside of php tags to make it executable. Write your script (php) code to get, parse (use mimeparser class for php) and save (insert etc.) email data and set file permissions to 0755 (This is very important).

You are good enough to go with it now, i think. Do post comments as they inspire me a lot. Thanks!

6 thoughts on “Setting up a catch-all email forwarder and parsing emails using…

  1. Yeah. Still have a problem though. The parser doesn’t seem to split the mails up. Let’s say I send two mails at the exact same time, and store then in a database, it will store twice the same message to the same address.
    That may not sound very clear, but in other words, I send an email to two different receivers at the exact same time and it will store the first mail only.

    I downloaded your example script and noticed you were splitting them up too, do you have this issue?

    If this wasn’t clear, please let me know, I’ll try to explain better.

  2. Actually, I found out why it was doing this. It’s because the script add a space after the ending PHP tag, so the script was outputting something (a space in my case) and was causing the mail to bounce back to the sender.

    Working perfect now :)

  3. Well, I have set an email by default (to that PHP script), so the “to” address can’t be wrong really can it?
    I am getting this error on each domain I have tried and from each email I sent. Doesn’t matter what the script does, I get this message even if the “mail.php” script is empty.

    But everything is working though, I’m actually able to read this mail from my script and parse it etc…

    I guess it’s some settings somewhere I missed.

    Thank you for your reply!

  4. Hey.

    Thank you for this, it’s awesome. I didn’t find many resource on the Internet on how to achieve similar things.
    Anyway, that’s really cool of you.

    Though, I have one question. I am using cPanel (just like you) and I have set a default email and PHP is catching it so, so far, no problems. My problem is that it sends an email to the send saying this message could not be forwarded. Well, this is the exact message I am getting:


    This message was created automatically by mail delivery software.

    A message that you sent could not be delivered to one or more of its
    recipients. This is a permanent error. The following address(es) failed:

    pipe to |/home/xxxxxxx/public_html/mail.php
    generated by frank@xxxxxxx.net

    Have you encountered this issue already? I know this article is a little “old” (old for the Internet) so I don’t really expect a reply, still I’d be surprised in a good way if you did.
    Thank you again for this article.

    Regards.

    1. @Bencori – Hi, i did not face such issue. The issue “A message that you sent could not be delivered to one or more of its..” usually occurs when sending of emails fail due to incorrect “to” email address. However your “from” address was correct that’s why you receive this message. Anyways emailing server is working fine that’s why you receive the message.

      I assume that you are “catching all” requests to a PHP script and then trying to send emails from the script which is failing. Are you? If so, please check your send email script and also try with different “to” address.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>