Tag Archives: spam

Mail, Open Source

Absorbing 1,000 emails per day

Some people say email is dead. Some people say there are “email killers” and bring up a bunch of chat and instant messaging services. I think those people communicate far too little to understand how email can scale.

I receive up to around 1,000 emails per day. I average on a little less but I do have spikes way above.

Why do I get a thousand emails?

Primarily because I participate on a lot of mailing lists. I run a handful of open source projects myself, each with at least one list. I follow a bunch more projects; more mailing lists. We have a whole set of mailing lists at work (Mozilla) and I participate and follow several groups in the IETF. Lists and lists. I discuss things with friends on a few private mailing lists. I get notifications from services about things that happen (commits, bugs submitted, builds that break, things that need to get looked at). Mails, mails and mails.

Don’t get me wrong. I prefer email to web forums and stuff because email allows me to participate in literally hundreds of communities from a single spot in an asynchronous manner. That’s a good thing. I would not be able to do the same thing if I had to use one of those “email killers” or web forums.

Unwanted email

I unsubscribe from lists that I grow tired from. I stamp down on spam really hard and I run aggressive filters and blacklists that actually make me receive rather few spam emails these days, percentage wise. There are nowadays about 3,000 emails per month addressed to me that my mail server accepts that are then classified as spam by spamassassin. I used to receive a lot more before we started using better blacklists. (During some periods in the past I received well over a thousand spam emails per day.) Only 2-3 emails per day out of those spam emails fail to get marked as spam correctly and subsequently show up in my inbox.

Flood management

My solution to handling this steady high paced stream of incoming data is prioritization and putting things in different bins. Different inboxes.

  1. Filter incoming email. Save the email into its corresponding mailbox. At this very moment, I have about 30 named inboxes that I read. I read them in order, top to bottom as they’re sorted in roughly importance order (to me).
  2. Mails that don’t match an existing mailing list or topic that get stored into the 28 “topic boxes” run into another check: is the sender a known “friend” ? That’s a loose term I use, but basically means that the mail is from an email address that I have had conversations with before or that I know or trust etc. Mails from “friends” get the honor of getting put in mailbox 0. The primary one. If the mail comes from someone not listed as friend, it’ll end up in my “suspect” mailbox. That’s mailbox 1.
  3. Some of the emails get the honor of getting forwarded to a cloud email service for which I have an app in my phone so that I can get a sense of important mail that arrive. But I basically never respond to email using my phone or using a web interface.
  4. I also use the “spam level” in spams to save them in different spam boxes. The mailbox receiving the highest spam level emails is just erased at random intervals without ever being read (unless I’m tracking down a problem or something) and the “normal” spam mailbox I only check every once in a while just to make sure my filters are not hiding real mails in there.

Reading

I monitor my incoming mails pretty frequently all through the day – every day. My wife calls me obsessed and maybe I am. But I find it much easier to handle the emails a little at a time rather than to wait and have it pile up to huge lumps to deal with.

I receive mail at my own server and I read/write my email using Alpine, a text based mail client that really excels at allowing me to plow through vast amounts of email in a short time – something I can’t say that any UI or web based mail client I’ve tried has managed to do at a similar degree.

A snapshot from my mailbox from a while ago looked like this, with names and some topics blurred out. This is ‘INBOX’, which is the main and highest prioritized one for me.

alpine screenshot

I have my mail client to automatically go to the next inbox when I’m done reading this one. That makes me read them in prio order. I start with the INBOX one where supposedly the most important email arrives, then I check the “suspect” one and then I go down the topic inboxes one by one (my mail client moves on to the next one automatically). Until either I get overwhelmed and just return to the main box for now or I finish them all up.

I tend to try to deal with mails immediately, or I mark them as ‘important’ and store them in the main mailbox so that I can find them again easily and quickly.

I try to only keep mails around in my mailbox that concern ongoing topics, discussions or current matters of concern. Everything else should get stored away. It is hard work to maintain the number of emails there at a low number. As you all know.

Writing email

I averaged at less than 200 emails written per month during 2015. That’s 6-7 per day.

That makes over 150 received emails for every email sent.

Mail, Web

groups.google.com hates greylisting

3 Comments

Dear Google,

Here’s a Wikipedia article for you: Greylisting.

After you’ve read that, then consider the error message I always get for my groups.google.com account when you disable mail sending to me due to “bouncing”:

Bounce status Your email address is currently flagged as bouncing. For additional information or to correct this, view your email status here [link].

Following that link I get to read the reason:

“Google tried to deliver your message, but it was rejected by the server for the recipient domain haxx.se by [mailserver]. The error that the other server returned was: 451 4.7.1 Greylisting in action, please come back later”

See, even the error message spells out what it is all about!

Thanks to this feature of Google groups, I cannot participate in any such lists/groups for as long as I keep my greylisting activated since it’ll keep disabling mail delivery to me.

Enabling greylisting decreased my spam flood to roughly a third of the previous volume (and now I’m at 500-1000 spam emails/day) so I’m not ready to disable it yet. I just have to not use google groups.

Update: I threw in the towel and I now whitelist google.com servers to get around this problem…

Blogging

a 20 to 1 spam to comment ratio

4 Comments

It has been a little over 1500 days since I started this (wordpress’ed) version of my blog. During this time, I’ve posted entries, people have submitted comments and most of all there have been spammers posting “comments”.

During these 1500 days I’ve posted over 600 blog entries. Roughly one entry every 2.5 days. We can see that my visitors aren’t that talkative in comparison as I’ve received some 550 comments in total to my blog posts.

10,000 spam comments have been submitted. That means roughly 20 times more spam than legitimate comments. The world can indeed be a sad place at times! 🙁

Mail

generic opt-in spam lists don’t exist

The last couple of days I’ve received a number of Swedish spam emails and I started digging up the Swedish companies behind them. The vast majority of all spams I get and have gotten during the years are English, so the Swedish ones stand out and they are a relatively new thing.

There seems to be a range of companies that now offer “email marketing” as a service to other companies. And there are lots of companies apparently willing to use such services. The other day the somewhat respected ISP company Crystone for example went ahead and spammed “a few hundred K recipients (link to a Swedish-speaking forum). I’ve long been annoyed by the repeated spam mails I get from the company Jajja, which apart from being in the snake oil business (SEO) seems to be a legitimate business that wants to be taken seriously. Of course, they have a shady history of bad business ethics (link to Swedish article about Jajja doing blog-comment spamming in 2007).

A can with spamCrystone’s excuse for their spam outburst was that they had bought this list of “verified” and “opt-in” addresses (from big-time spammer company mailcom.se) so they were quite surprised when large amounts of people started complaining and whining about their spam. mailcom.se, unsurprisingly, on their site boast to also have Jajja as customers. I have emailed mailcom.se and complained in strongly worded terms. I expect no response or effect.

Hejsan

Detta är ett av tjogtals (hundratals?) spam email jag fått från er. Ni har hittat/köpt denna email-address genom web-scraping och ni och era kunder är inget annat än spammare. Det är illegalt i Sverige och att betrakta som ett vedervärt sätt att försöka marknadsföra någonting.

Fy skäms!

The above is the email text I sent. It could be translated into English like:

Hello

This is one of the many (hundreds?) spam emails I’ve received from you. You found / bought this email address by web-scraping and you and your customers are nothing but spammers. It is illegal in Sweden and to be regarded as a horrible way of trying to market anything.

Shame on you!

Newsflash: there is no such thing as a blanket list with verified and opt-in email addresses. You may get people to opt-in for a particular and well explained purpose, but nobody ever asked anyone if they wanted to get stupid market emails from Crystone without compensation. Who would have opted-in to something like that?

Legality? People here in Sweden are quick to point out that sending market emails to companies and other business is not illegal here. Although, as is easily proven, these guys don’t know who they target as their list clearly is created by old fashioned web scraping techniques and they send to anyone, individuals and companies – without discrimination. Besides, my biggest complaints against spam is that it is a nuisance and a pain, if it is illegal or not is not the biggest concern to me. Spam is spam no matter what.

I’ve also explicitly tweeted about the spam service provided by quicknet.se. They’re at least somewhat open about it and add a header in their outgoing mails claiming them to be from “QuicNet_AB” (notice how the letter k is absent). I’ve received several spams via their domain gallerian.org so there’s no doubt who’s behind them. These mails also have ended up targeted to email addresses that are without any doubt harvested from the web. An employee of quicknet responded to me (in Swedish), apparently surprised by my allegations but I’ve received no further info. But frankly, I don’t care what excuse they can come up with. It will only be something lame as this is not a mistake.

Other seemingly popular Swedish spam companies include epostservice.se/com, epostarna.se and so on. I wish more people will react on the spam and object to the companies that buy these services (in good faith or not) and to the companies that provide these services. Tell them it’s all spam, no matter what excuses they can figure out!

PS. Yes, this is the same Crystone I’ve written about before…

Mail

Email asking for my products

In my mini-series of strange mails I receive, here’s another one:

Subject: Product Request

Hello,
I am interested in purchasing some of your products, I will like to know
if youcan ship directly to SPAIN , I also want you to know my mode of
payment for this order is via Credit Card. Get back to me if you can ship
to that destination and also if you accept the payment type I indicated.
Kindly return this email with your price list of your products..

I assume I’ll never figure out what products he speaks of, or how on earth he ended up sending me this… I’ll admit I was tempted to make up some “interesting” products to offer.

Update: I was informed that this is probably “just” another online fraud attempt. How boring.

Mail

Spammers now subscribe

2 Comments

During several years I’ve been setting mailing lists I admin to only accept posts from subscribers iA can with spamn order to avoid having to deal with very large amounts of spam posts.

While that is slightly awkward to users of the list, the huge benefit for me as admin has been the deciding factor.

Recently however, I’ve noticed how this way to prevent spam on the mailing lists have started to fail more and more frequently.

Now, I see a rapid growth in spam from users who actually subscribe first and then post their spam to the list. Of course, sometimes spammers happen to just fake the from address from a member of a list – like when a spammer fakes my address and sends spam to a list I am subscribed to, but it’s quite obvious that we also see the actual original spammer join lists and send spam as well.

It makes me sad, since I figure the next step I then need to take on the mailing lists I admin is to either spam check the incoming mails with a tool like spamassassin (and risk false positives or to not trap all spams) and/or start setting new members as moderated so that I have to acknowledge their first post to the list in order to make sure they’re not spammers.

Or is there any other good idea of what I can do that I haven’t thought of?

IT Politics, Mail

Explanation for hjsdhjerrddf.com domains

In case you’ve checked some of your spam mails recently you might’ve discovered how a large amount of them include links to sites using seemingly very random names in the domain names. Like hjsdhjerrddf.com or qwetyqfweyqt.com and so on. Hammering-the-keyboard looking names.

The explanation behind these is quite simple and sad: ICANN allows for a “tasting period” before you pay for the domain. Thus spammers register all sorts of random names, spam the world with mails referring the users to these domains and then they return the domain names again before they’ve paid anything, and go on to the next names.

With a large enough set of people and programs doing this, a large amount of names will constantly be kept in use but not paid for and constantly changing owners.

Conclusion: wherever there’s a loophole in the system, someone is there to exploit it for the purpose of sending spam.

Mail, Open Source

My best spam rules right now

I’ve already before mentioned my antispam setup, but today I just ran a little check on my “hispam” mailbox (the spams with so high spam points that I never even bother to check them for false positives), 43MB of 7900+ spams (received during ~40 hours), to see which ones of my own handicrafted rules that get triggered the most. I use a set of 40+ custom spamassassin rules to help it trigger more mails as spam, since some of the very short mails seem to be hard to catch otherwise, and some of the mails are in many ways looking like mail I would normally get.

Anyway, my top-10 rules are:

  1. 1624 6.0 DS_BODY_DRUGBRAND      BODY: mentions drug brand
  2. 1428 6.0 DS_SUBJECT_DRUGBRAND   Subject mentions drug brand
  3. 828 6.0 DS_FROM_HAXX     spoofed haxx.se address
  4. 769 4.0 DS_BODY_DISCOUNT    BODY: mentions percent discount
  5. 745 4.0 DS_SUBJECT_DISCOUNT   subject mentions percent discount
  6. 415 2.1 DS_TO_OWNER   To contains -owner
  7. 200 6.0 DS_BODY_NODOCTOR  BODY: mentions “no doctor”
  8. 195 2.0 DS_MAILER_THEBAT  sent with the bat
  9. 189 6.0 DS_BODY_DESIGNBRANDS  BODY: mentions designer brand(s)
  10. 158 3.0 DS_BODY_REPLICAS  BODY: speaks of replicas

The first number is number of hits. The second is the “spam points” I assign a match. Then there’s the name of the rule and my description for it. The “spam points” can best be seen relative to the other rules, as what makes a single mail a spam in the end involves multiple factors that aren’t shown here.

Mail

Mail turned unreliable

I’ve always been proud of my ability to read and respond to email in a swift and reliable manner. I read and write emails every day, and most days I read mails more or less immediately as they land in my inbox.

However, during the recent year or so I’ve noticed that I’m no longer a reliable mail recipient. The amount of spam I get has made me tighten the screws so hard I get my share of false positives. The kind of mails that I need to rescue from my spam bin as they will otherwise suffer the death by delete. But how many do I miss? How often do I lose legitimate mails?

On some of the mailing lists I participate in, the spammers have started to send posts with my email in the From: field (circumventing the subscribers-only limitation), leading to me having to set my own mails as moderated to prevent spam to get posted… 🙁

Blogging, Open Source

WordPress quirks and edits

There’s no secret I’ve had my share of gripes with WordPress and here comes two more:

I can’t upload images at the moment! I run the “plain” wordpress package in Debian testing and when I try to upload an image using the fancy new ajax way in 2.5, it just sits there for a while and it seems it receives the file but I don’t get the UI up that I believe I should get when the upload is completed… so I can’t confirm the upload etc so it instead it gets discarded!

I’m suffering a bit from trackback spam so I installed a plugin named Trackback Validator to help me reduce the manual work of denying them. It seems to work rather well so far in that I now no longer have to mark very many comments (trackbacks appear as comments within WordPress) at all, but the annoying part is that even though the validator unvalidates the trackbacks I still get information mails sent out to me about them! I’ve now also enabled the Akismet plugin so let’s see what happens. Of course simply disabling trackbacks is an option that I’ll use if this doesn’t work good enough.

A funny side-effect with installing and enabling Akismet was that all of a suddent I could access comments previously marked as spam, and thus I could undo the damages from my accidental mark-as-spam-hiccup the other day!

While playing around with plugins, I also installed a gravatar plugin that shows gravatar-images for users on comments, and I installed a plugin that will automatically set my timezone correctly even when DST changes – which WordPress can’t do by itself!

Then all of a sudden when I poked around (too much) I managed to somehow ruin the background image I use a the top of all pages on my blog. Somewhat I got a gradient there instead, which indeed is what the theme supports (the theme I use is of course a standard one but I have done some minor edits of it). Took me a while to manage to get rid of the gradient and get back image back… I had to resort to editing the PHP file for the theme!