Bad Words

You have a list of "bad" words or phrases that might indicate an offensive or off-topic message. Initially, this list will contain over 100 words that we have chosen. Each word in the list carries a certain penalty value, as indicated by the number beside it. When you click "Save Changes" the system automatically sorts the list, and puts all words into upper case. For example,

        1-800-, 12
        CASINO, 8
        DAMN, 5
        OFFER EXPIRES, 10

You should modify this initial set of words and values to suit the subject of your mailing list, and your own policy. Some words are there to catch typical SPAM. Others are there to catch foul language. Depending on the nature of your mailing list, you might want to allow foul language, and if your list happens to be about gambling you would obviously delete the entry for the word "CASINO". Software developers might want to enter passwords, registration codes, etc. that some people on the list know, but others are expected to pay for. With ListFilter, you have a program running 24 hours per day, reading every message, ensuring that critical information does not get leaked, either accidentally or deliberately.

The moderation program checks every line in every message, looking for bad words. Bad words are matched without regard to upper/lower case. So, CASINO would match CASINO, Casino, casino, casinos etc. When a bad word on your list is 3 characters or less, it must match a complete word, not just a substring of a longer word. Otherwise short words might trigger too many spurious matches.

The score goes up with each occurrence of any bad word, but the penalty is reduced by 20% for a second occurrence of the same bad word, and another 20% for the third occurrence, and so on. Eventually the penalty (rounded to the nearest integer) will reach zero. This helps to reduce false alarms. It's more significant to see two different, equally bad, words appearing in a message, than it is to see two occurrences of the same bad word.

For example, 3 occurrences of CASINO would generate a score of 8 + 6 + 5 (rounded off to the nearest integer), rather than 8 + 8 + 8 as you might expect.

So, for example, a message that contained CASINO and OFFER EXPIRES and 1-800- and another CASINO would be scored as:

    8 + 10 + 12 + 6 = 36

If the threshold was 30, the message would not be approved. It would be forwarded to you for evaluation.

Note that this scoring system is more subtle than simply saying that any message containing CASINO must be blocked, or any message with an 800-number must be blocked. We'll soon see how even more subtlety can be added to the system. The goal is to create an artificially intelligent moderator that can make the right decision in almost all cases, while letting you have the final say about rejecting messages.


Latest Comments