13 Feb

2012

SiteGround goes SPAM FREE

When I say SPAM, you will most probably think of the annoying unwanted emails INCOMING to your own Inbox, and this is definitely one of the SPAM aspects we have been fighting for a long time and which I am proud we have already minimized very successfully. However, in this post I would like to address another aspect – the OUTGOING SPAM: how it affects you and what is the newest solution we have applied to it.

Since I started dealing with hosting (and probably many years before that), outgoing SPAM has been one of the most troublesome aspects for a hosting company. It might sound strange to you, but the spammers are causing hosting companies (and you, our customers) a lot of troubles, headaches and even migraines. How? I will explain in details below.

There are so many shared hosting servers out there, configured to be able to send email. Each of them hosts thousand of websites, which according to a rough personal estimation, in more than 50% of the cases run a heavily outdated apps like WordPress 2 or Joomla 1.0, with outdated plugins and themes. This situation creates infinite possibilities for the hackers to intrude into a vulnerable account and use the servers’ email sending capacity to produce a vast amount of emails concerning Viagra, Cialis and cheap Rolex watches on a daily basis.

When such a scenario occurs the hosting company would usually know about the emails being sent out within 10 minutes or less, simply because trying to send thousands of emails causes serious CPU load to the server. The server admin sees that the load level goes high and logs in to see what causes this, he sees the spam and cuts it. But it is too late as 10 minutes is a lot of time for spamming. On a normal Dual Processor Server with stable network connectivity, 10 minutes would be enough for sending at least several thousands or several tens of thousands of emails. At that point the big troubles begin – your server IP gets blacklisted at various RBL lists. And now none of your users is able to send email messages any more unless the IP is white-listed again. The users of course are not really happy with that misfortunate outcome and start calling you, posting you trouble tickets, angry twitter comments, etc. while you anxiously try to get the IP address white-listed again. Which of course is not an easy task to do, since RBL lists are not maintained for profit and they don’t have the best support out there. In fact, we’ve seen cases when whitelisting an IP address from a specific RBL took more than 30 days and you would definitely not be happy not being able to send an email for 30 days…

Throughout the years we’ve been on the fight with this problem continuously:

1. We have built an automated monitoring system that allow our 24/7 admin-gurus to react to a high CPU usage and stop SPAM in 45-60 seconds, which is amazingly fast, but … unfortunately still not fast enough, as our servers are capable of sending a lot of emails for that time and it’s usually enough to get us and our users blacklisted.

2. Seeing that the speed of solving the problem does not help in many cases, we turn our efforts to preventing the problem from appearing at all. We have invented an in-house solution based on SpamD and spam scores, and invested a lot of time to try to train it properly to stop mail messages, that are likely to be SPAM to go out of our servers. However, having access to data about our servers sending behavior only was not enough to train the tool fast enough.

So, at that point of our fight we met SpamExperts at a conference. They had a booth next to ours and this turned out to be an important meeting. Apart from the fact they are a great bunch of people, they had what we did not – a system that filtered more than 50 million mail inboxes and is already so well developed to recognize outgoing SPAM, that we would hardly catch up with our in-house efforts based on a considerable lower number of mails. We were also intrigued by the architecture of their service and the way it works. So, we tested the solution for several months and it showed great results stopping the outgoing SPAM and preventing our IPs from being blacklisted in more than 95% of the test situations.

So having finally found something we believe has the potential to solve our problem with the outgoing SPAM, we were even more excited to see that the same service will address the incoming SPAM too in the same efficient way. Currently, we have our own in-house build SPAM protection already working quite effectively. Additionally, we also provide an extra protection with Mailfoundry Spam Appliance to the interested customers. However, Mailfoundry has one serious flaw that prevented us from providing it to everyone – it was not easily scalable. On the other side, one of the greatest advantages of the SPAM experts solution is that it operates in a cloud environment and it is extremely easy to add and remove accounts to it. In addition to being much more scalable, the SPAM experts solution also has a very cool cPanel integration, which will allow all our clients to manage their spam filtering, quarantine zone and see various stats on their email traffic.

All these facts lead us to an easy decision: we will start using the SPAM Experts solution. We have already started to deploy it on SiteGround Shared, SharedPlus and Semi-Dedicated plans, applying its protection to both incoming and outgoing mail messages. The solution will be used in addition to our current in-house designed tools and will gradually replace the MailFoundry service. And last but not least – it’s going to be completely free of charge for you, the customer.

Best,

Tenko
The SiteGround Mastermind

Author: Tenko Nikolov

The SiteGround Mastermind

For the last few years Tenko Nikolov has been one of the masterminds behind the success of SiteGround. He has come up with multiple successful strategies for overcoming technical problems and has achieved real business results for SiteGround. His vision and skills have made SiteGround a leading host in terms of technology and platform reliability.

Comments (10):

  1. Adewale Paul says:

    Words cant express my joy for this recent development in siteground, this is no adulation, you guys rocks big time.U are just so amazing sometimes i cant help but wonder what the catch is only to discover there is no hidden plan. You have saved alot of us from this.I just hope people are wise enough to move their web to yours cus you guys are just awesome. You can always count on me to bring in more referrals.
    Great Job once again

  2. Phil Seghers says:

    Hi there,

    It would be great not to have heaps of unwanted emails.
    There are times when I recieve heaps over a few days.

    I am very pleased with the support & operation of my Siteground hosting, keep up the good work
    Regards, Phil

  3. Fletchus says:

    It would have been nice to have had advance warning on this being being inserted into the e-mail chain. It caused me days of untold grief trying to figure out what was going on with my client’s e-mail. For example, the law firm that had every message from its website contact forms automatically quarantined for weeks because it “looked like spam”.

  4. Mike Doyle says:

    While this is indeed and admirable feat, I have to say I was quite annoyed when my incoming emails were being blocked as SPAM. This was done “silently” on 2/13 and that in itself annoyed me. Some of us rely on email for out core business. I use it for voicemail, for example. When my outgoing server is blocked by your SPAM solution and there is no way for me to add it to my “allow” list then I have a problem. Implementing a solution is easy, but insuring that nobody’s business is interrupted by it is entirely different. It seems that SiteGround has selfish interests and discounts the potential damage to their customers. Please, in the future, think at least twice before acting once.

    • Marina says:

      Hello Mike Doyle, and hello Fletchus,

      We integrated the SpamExperts solution in our services since it is a good product that resolves issues with SPAM very efficiently. A single solution cannot achieve 100% accuracy, but in our opinion SpamExperts is doing times better than anything else we know of. Furthermore, it allows lots of fine-tuning and filtering in order to further increase its accuracy.

      In general, hosting companies do not offer spam experts for free but make their customers pay additional fees to use it. We decided to invest in this software and make it free for all our shared servers because we truly believe that its long-term benefits will outnumber the short-term setbacks. In any case, we are very sorry if there is some inconvenience caused after the deployment of the software. We are eager to make all the necessary modifications to spam experts’ solution and work with you individually in order to make it better suitable for you.

      Thank you for pointing out some of the problems you have experienced.

      • T Vance says:

        I appreciate the effort as well but there were simply too many emails that were blocked or delayed, even emails sent from within our domain. I like the control and reporting ability that SE has but the emails I have missed already have cost us business. I have disabled SE from our domain hoping that the problems go away.

        While I was disabling SE I noticed that the MX records don’t appear to be customizable anymore?!! Unless you are a google apps user there doesn’t appear to be a way to set them anymore.

  5. Nikolay says:

    Hello Vance,

    First of all I would like to thank you for your feedback. Indeed we’ve disabled the cPanel native MX management tool as it turned out to be incompatible with our mail service setup. It was causing more problems, rather helping. However, simply disabling it was not the best solution and that is why we developed alternative MX plugin, which you mentioned already.

    Currently, this MX tool is limited to some basic options, like setting up Google MX records, reverting back to the Spam Expert defaults and to disabling SE at all. We plan to extend the functionality of this tool based on the customers demand.

    Regards,
    Nikolay T
    CTO @ SiteGround.com

  6. Sami Vapalahti says:

    I might add that I have got already several returns of trying to send email from my account and I do not like it. I propose you should concider some frequency in the filter. How likely will spam mail be sent 1/hour? I do not think very likely. Maybe monitoring mass mails or emails send at high frequency are much more likely to be spam. In any case I would love to know why is my rare emails concired spam, and how can I prevent that from happening.

    Sami

    • Nikolay says:

      Hi Sami,

      Spam Experts obviously detects some of the emails you send as spam. The mail sending interval is not an issue here. As long as the system detects an email’s subject or body as spam, it will filter it. Please also note, that the reason for message block is not only limited to spam. Some message are being blocked if there is detected malicious code within them or URL to such.

      I asked our system administrators to check the Spam Experts outgoing mail log for your account for any filtered messages and for the last week or so they found only 2 emails that were blocked on the 6th of April. They released them and this way the system should “learn” not to filter such messages any more.

      If, however, you still get some emails blocked by Spam Experts, please post a support ticket through the SiteGround Help Desk so that our support team can review your case thoroughly and make sure your issues are permanently fixed.

      Regards,
      Nikolay T
      CTO @ SiteGround.com

  7. Mirsada says:

    Hi

    I am beginner and opened account on siteground because joomla give good reviews about this hosting. I put PB Booking software and it worked very well on greengeeks hosting. In siteground every message is spam and this software can not work. It seems that this is a waste of money and an entire day in an attempt to solve the problem.

    Mirsada

Favorite Tweets

Facebook

Archive