NinerNet Communications™
Blog

Corporate Blog

Mail server in and out of capricious blacklist

10 March 2020 02:33:48 +0000

As you’re aware, we work hard to ensure that our mail servers do not get into blacklists. On the rare occasion that one of our IP addresses is blacklisted, we investigate the cause of the problem, fix the problem (often a client with a compromised machine) and (if possible) try to have our IP address removed from the blacklist. Often though, manual removal from the blacklist is unnecessary, as modern, well-maintained blacklists are automated, and offending IP addresses are removed very soon after they no longer show any signs of sending spam.

It’s not often any more that we run into old-style blacklists — blacklists that are poorly maintained, that blacklist huge swathes of the Internet, or that offer no discernible removal process — but there are still some of them out there. Not many are used by mail servers that accept email on behalf of any sizeable number of users, but we have run into one that happens to fit that exact trifecta: urbl.hostedemail.com.

This blacklist is used by Hostedemail(.com), a subsidiary of OpenSRS/Tucows. Good luck getting to their website though, as one doesn’t exist. Their email hosting service is a white-label service sold by their resellers, and they don’t even have a way for other mail server administrators to contact them, to search their blacklist or ask to be taken out of it.

Thankfully though, we are still hanging onto our own long-established reseller account with OpenSRS, and we contacted them about this block of our (non-resold) primary mail server (NC036). We first did this in February when we noticed that email from some clients was being blocked with this error message:

host mx.DOMAIN.com.cust.a.hostedemail.com[216.40.42.4] refused to talk to me: 554 5.7.1 Service unavailable; Client host [178.62.195.26] blocked using urbl.hostedemail.com; Your IP has been manually blacklisted

(It was the reference to being “manually blacklisted” that really got our attention, as this is a hallmark of the aforementioned poorly maintained blacklists.)

OpenSRS responded quickly, and we were removed from the blacklist within about eight hours. But we were surprised to see, a couple of weeks later in March, that we were blacklisted again, so we contacted OpenSRS yet again. The response this time was much slower, but we have again been removed. This time, however, we pressed for an explanation for the block, as we are not listed in about 300 other blacklists that are more widely used. This is part of their response:

I am just replying back on the RBL listing you inquired about and I can confirm the IP was once again de-listed but I did get some additional information for you as requested. I needed to do a bit of checking but the IP 178.62.195.26 is provided by RIPE Network Coordination Centre, the IP assigned to the user by the hosting provider carries the reputation of the rest of the CIDR. The nature of VPS/Shared IPs is to be disposable …. But of course for the time being we have de-listed the IP but assuming nothing changes its [sic] likely it will be listed again in the future.

This kind of outdated thinking is another of the hallmarks of old-style blacklists: blacklisting half of the Internet based on some outmoded way of thinking that died off around the end of the twentieth century. Essentially, Hostedemail.com is blacklisting all IP addresses in major data centres around the world, which is very counterproductive for their own customers.

We’ll be contacting individual clients whose emails were blocked by this blacklist to point them to this post, and we recommend that if your email is blocked with the above message you contact your correspondent by some other means to advise them to move to a more enlightened mail service provider.


Update, 2019-03-19: Our primary mail server is again blacklisted by this one mail provider in the world out of about 300 major blacklists we have checked. OpenSRS/Tucows/Hostedemail warned us this would happen, so we’re not surprised. We can take no further logical action against an illogical practice. We’re sorry to those clients who are affected, but we again suggest that you tell your correspondents to move to an email service provider that doesn’t run their mail servers based on practices from the last century.

Spam and virus filtering on the mail server

11 October 2018 15:15:22 +0000

Over the last five months we’ve been monitoring the effectiveness of the anti-spam systems on server NC036 with a view to setting the point at which emails are considered by the system to be spam. We have slowly lowered the cut-off point from the default of 6.2 to 3.0, and have found that at 3.0 the rate of legitimate email caught in the filter rises sharply. Therefore we have now set the default, server-wide level at 3.5. At this point we’re blocking about one thousand to fifteen hundred spams a day, and anywhere from a handful to a few dozen viruses a month.

You can set a different cut-off point for spam to your domain(s) as follows:

  1. Log into the mail server control panel.
  2. Click “Domains & Accounts”.
  3. Click the domain you want to manage.
  4. Click “Spam Policy”.
  5. Enter a different number in the “Classify mail as spam when score is >=” field.
  6. Click the green “Save changes” button.

In short, the lower you set the score the more spam is caught, but the greater the likelihood of legitimate email being classified as spam. Conversely, the higher the score you set the less spam will be caught and the lower the likelihood of legitimate email being classified as spam.

You can also manage other aspects of the spam filter on this page, but we recommend that you do not. The server-wide defaults are to enable all four checks (spam, virus, bad headers and banned files) and to quarantine spam and viruses. If you want to allow any of those four classes of undesirable emails through on your domain that’s your call, but you take full responsibility for the results. The results include everything from annoyance to compromised machines, devices and accounts. NinerNet does charge for time spent recovering and cleaning up compromised accounts.

Please note that the spam and virus filters monitor both incoming and outgoing email.

We strongly recommend, now that we have finished our evaluation, that you conduct your own evaluation of the situation with undesirable email on your own domain or domains. Once logged into the mail server control panel, please navigate to System -> Quarantined Mails. There you will find spam and virus emails to and from your domain(s) for approximately the last week. As mentioned above, if you find that too many legitimate emails are being classified as spam, you have two options: 1) Increase the score at which messages are considered spam, and/or 2) Whitelist any legitimate senders or domains that consistently receive high scores. To whitelist a “sender” (a single email address) or a domain or a domain and all of its sub-domains, follow these instructions:

  1. Log into the mail server control panel.
  2. Click “Domains & Accounts”.
  3. Click the domain you want to manage.
  4. Click “White/Blacklist”.
  5. Follow the instructions on the right of the page to add records to the appropriate whitelist, incoming or outgoing.

Please note that it might be tempting to add something like @yourdomain.com to the outgoing whitelist (thereby whitelisting all addresses on your domain), but we strongly advise you not to. If you do, and a machine on your network is infected with a virus or is compromised and starts spamming, the system will follow your instructions and let it all through. Please see above about our fees for cleaning up after a mess like this. The emails will likely be blocked on the receiving server anyway, and your domain possibly blacklisted. You don’t want you domain (or our mail server) blacklisted, so not whitelisting all of your users is a defence against getting your domain (and our mail server) blacklisted.

Something else to note is that it’s fairly pointless to blacklist spammers and virus senders. If you blacklist bob@example.com because he sent a virus that the virus scanner caught, you’ll also block the legitimate email he sends once he cleans up his machine and sends you an email to apologise. Similarly, spammers rarely use the same email address or domain more than a few times, so you’ll just be filling your blacklist with a lot of crap. Of course, if a persistent spammer keeps getting through the spam filter, then go ahead and blacklist them if they’re actually using the same email address or domain.

Please monitor your quarantine on a regular basis so that you notice trends and compensate for them. With our evaluation ended we will only occasionally monitor the quarantine to make human judgement calls about letting some emails through, as we have been doing over the last five months.

It is worth noting here a couple of points. One is that no spam filter is perfect. During our evaluation we have seen spam come in that was scored less than 3.5, and so will make it through the filter now that we have settled on a cut-off of 3.5. Another is that some legitimate email from senders hosted on this server — i.e., you and your colleagues and employees — has been scored above 3.5 and so has been (or will be) quarantined instead of being delivered to the sender’s mail server. This is why you need to keep an eye on the quarantine for the domains under your account, and if necessary release legitimate emails for delivery. This is how you release emails:

  1. Log into the mail server control panel.
  2. Navigate to System -> Quarantined Mails.
  3. Select the legitimate email or emails.
  4. At the bottom of the page select “release selected” from the “Choose Action” drop-down list.
  5. Click the green “Apply” button.

The emails will then disappear from the quarantine and will be delivered to the recipients. You may also select one of the other three “release” options if you want to release the email and add the sender to your whitelist if their email is consistently being scored highly. As mentioned above, it’s generally a waste of time to select one of the blacklisting options; there’s also no need to manually delete items from the quarantine, as they are rotated out after about a week.

With respect to your own emails being marked as spam, there are some glaring spam markers that we’ve seen commonly used that you and your colleagues and employees can avoid by following these suggestions:

  • Don’t use blank subjects.
  • Don’t use ALL CAPITALS subjects. If you do, keep in mind that your method of trying to get the recipient’s attention might fail completely if your message is blocked as spam.
  • Avoid using very short subjects.
  • Avoid using “Dear xxxx” in your salutations. Email is a less formal mode of communication than letters, and opening an email with “Dear” is a classic spam marker and will give your email enough extra points that it could push it over the cut-off score, especially when combined with other spam markers listed here.
    • Update: Thanks to a client for pointing out that “Dear Bob” or “Dear Mrs. Smith” are not scored as badly as generic salutations such as “Dear sir”, “Dear madam”, “Dear investor”, “Dear home owner”, “Dear winner”, “Dear beneficiary”, “Dear friend”, “Dear you@example.com”, etc.
  • Don’t send blank emails with only an attachment.

Please note that we don’t read your email. This data is gleaned from the spam reports and the reasons that certain messages were blocked because they were classified as spam.

This spam filter is much better than what we had on the old email server, and now you have access to the information it contains and control over how it works. If you have any questions or concerns, please contact NinerNet support. Thank-you.

NinerNet home page

Subscriptions:

RSS icon. RSS

General Information:

This is the corporate blog of NinerNet Communications. It's where we post announcements, inform and educate our clients, and discuss issues related to the Internet (web and email) hosting business and all it entails. This includes concomitant industries and activities such as domain registration, SSL/TLS certificates, online back-up, virtual private servers (VPS), cloud hosting, etc. Please visit our main website for more information about us.

Search:

 

Recent Posts:

Archives:

Categories:

Tags:

accounts receivable apple billing branding cira contact information domain registration domain registry of canada domain renewals domains domain sales dot-ca domains dot-zm domains down time droc email facebook google happy hosting customers hosting transfer icann internet registry of canada invoices iphone iroc kwacha maintenance paying your bill paying your invoice quarterly kwacha rate review rates registrant transfers registrar transfers reputation scams search engine optimisation search engine optimization security seo service hours spam support transparency wordpress zamnet

Resources:

On NinerNet: