StupidFilter – Time to Fight Back the Imbeciles

A team of American scientists are developing the “StupidFilter” – an open-source filter software that will be able to detect “rampant stupidity” of web-content in written English. Similarly to the way spam recognizing software detects suspicious e-mails, the “StupidFilter” will look for pre-fed words or sign combinations that characterize stupidity, assigning particular tokens with different weights based on how often they occur in hand-picked examples of idiotic comments. The developers are using weighted Bayesian analysis along with some rules-based processing, similar to spam detection engines, in order to efficiently distinguish unacceptable messages among the submitted texts.
 StupidFilter - filtering comments YouTube (Credit: YouTube)
StupidFilter – filtering comments
from YouTube (Credit: YouTube)

The main challenge the developers face is the difficulty in distinguishing between silly or incomprehensible content and intentional spelling distortions, which could reflect sarcasm or emphasis. “Stupidity is not a binary distinction, but rather a matter of degree” – they say. For example, a comment like “this video is amazingggggg” should not be filtered because of its incorrect spelling, but something like “im firrsstt” will be tagged by the StupidFilter and the writer will receive a message demanding a change in phrasing before the comment can be posted. The pilot version of the StupidFilter will be able to filter comments with too much or too little capitalization, too many text-message abbreviations, excessive use of “LOL,” exclamation marks, and so on. 

The project is still in the design and analysis phase – scientists have already gathered over 255,000 real user comments, primarily from “YouTube”. These comments are ranked on a five-point scale by a web-based ranking system and the results are ‘fed’ into the StupidFilter’s database. This database forms the basis for the analysis of incoming comments in terms of diction, not content. An alpha source code release date is set for late December 2008, and the creators say they are likely to adhere to this deadline.     

Once the research is complete, the developers plan to release core engine source code for incorporation into content management systems, blogs, wikis and the like. Additionally, they look forward to developing a fully implemented Firebox plug-in and a WordPress plug-in. 

To the question “Isn’t filtering stupidity elitist?” the developers’ reply: “Yes. Yes, it is. That’s sort of the whole point. Effective textual communication requires at least some formal rigor, and we feel such rigor is worth encouraging and, at times, enforcing.”    

You can find more information about the StupidFilter on the official website.