Feedback Form
   
Add to Google
StupidFilter - Time to Fight Back the Imbeciles StupidFilter - Time to Fight Back the Imbeciles
Monday, December 10, 2007 - Sarah Gingichashvili
Home >> News >> Computer Technology
  Peralink
A team of American scientists are developing the “StupidFilter” – an open-source filter software that will be able to detect “rampant stupidity” of web-content in written English. Similarly to the way spam recognizing software detects suspicious e-mails, the “StupidFilter” will look for pre-fed words or sign combinations that characterize stupidity, assigning particular tokens with different weights based on how often they occur in hand-picked examples of idiotic comments. The developers are using weighted Bayesian analysis along with some rules-based processing, similar to spam detection engines, in order to efficiently distinguish unacceptable messages among the submitted texts.

 StupidFilter - filtering comments YouTube (Credit: YouTube)
StupidFilter - filtering comments
from YouTube (Credit: YouTube)
The main challenge the developers face is the difficulty in distinguishing between silly or incomprehensible content and intentional spelling distortions, which could reflect sarcasm or emphasis. “Stupidity is not a binary distinction, but rather a matter of degree” – they say. For example, a comment like “this video is amazingggggg” should not be filtered because of its incorrect spelling, but something like “im firrsstt” will be tagged by the StupidFilter and the writer will receive a message demanding a change in phrasing before the comment can be posted. The pilot version of the StupidFilter will be able to filter comments with too much or too little capitalization, too many text-message abbreviations, excessive use of "LOL," exclamation marks, and so on. 

The project is still in the design and analysis phase – scientists have already gathered over 255,000 real user comments, primarily from “YouTube”. These comments are ranked on a five-point scale by a web-based ranking system and the results are ‘fed’ into the StupidFilter’s database. This database forms the basis for the analysis of incoming comments in terms of diction, not content. An alpha source code release date is set for late December 2008, and the creators say they are likely to adhere to this deadline.     

Once the research is complete, the developers plan to release core engine source code for incorporation into content management systems, blogs, wikis and the like. Additionally, they look forward to developing a fully implemented Firebox plug-in and a Wordpress plug-in. 

To the question “Isn't filtering stupidity elitist?” the developers’ reply: “Yes. Yes, it is. That's sort of the whole point. Effective textual communication requires at least some formal rigor, and we feel such rigor is worth encouraging and, at times, enforcing.”    

You can find more information about the StupidFilter on the official website.
Related News D-Wave Demonstrates 28-Qubit Quantum Computer D-Wave Demonstrates 28-Qubit Quantum Computer Troubleshooting System for WiFi Networks Developed Troubleshooting System for WiFi Networks Developed

Related Pictures Fujitsu's Theft-Proof USB Fujitsu's Theft-Proof USB USB 3.0 to Come in 2008 USB 3.0 to Come in 2008

Other Articles Fujitsu S300 Scanner Review Fujitsu S300 Scanner Review Smart-Bombing Cancer Smart-Bombing Cancer



Comments & Replies (17)
da line bt stupidity & genious   (12/10/07 - 19:23 - by bullcutter)
what about sarcasm, what about short-hand, what about taking creative
license to prove a point? there are too many human elements easily
factored out by such a filter.
Stupid Filter   (12/10/07 - 19:51 - by WriterWriter)
Yay. Finally. I hope I can download the results to my blog, The Stupid
Files.
....Oh,I see what you mean by \"bt\" ...   (12/10/07 - 19:52 - by raptaj)
Case in point: your\'s. It is not hard to be THAT punctual while
proving a creative, sarcastic or nonsensical point.
ohnoes   (12/10/07 - 20:56 - by lawyer)
such a filter may pick up a substantial amount of legal language,
given that legal arguments typically conform to their respective
source of authority; ie, statutory language and elements within case
law are often repeated to death by both sides to an issue. i\'m just
sayin.
Excellent!   (12/10/07 - 21:44 - by Weed)
I look forward to this being implemented and the internet regaining
its intelligence.
A filter for "Stupidity"   (12/11/07 - 2:42 - by Patrizia Broghammer)
That will be something like:You can say whatever you like, as long as
it is what I like and think intelligent.
But who is going to filter the "intelligent" comments?
Awesome idea!   (12/11/07 - 2:52 - by Festus)
You could reduce the whole internet into one file that would fit on a
single flash card!
I see this as a personal victory   (12/11/07 - 4:43 - by Alfonso Gerniou)
It is about time that someone stood up to these stupid stupid people.
I once listened to these people's taunts and it put me in jail.

I have no time for these stupid and I hope they die.
@ Alfonso   (12/11/07 - 6:13 - by Festus)
What happened? Your posting was blank.
this doesnt seem like it would work   (12/11/07 - 9:41 - by Sphacelate)
what if you fictitiously quote something stupid to make a point.
\"penises smell like black people\" thats stupid im making an example
stupid filter   (12/11/07 - 9:49 - by bob brownlow)
it would be nice if this could be applied to the science channel and
to text books written by the scientists. filtering shit should be left
to the perpetrators.
Another application   (12/11/07 - 12:29 - by Aaron)
Maybe they should apply the filter to the Bible. Well, at least try.
I am sure it would just be easier to chuck it in the trash.
Foolish   (12/15/07 - 17:54 - by Nick)
Hah, calling them "scientists" is quite charitable. What do they
think? Programming in a method of calling trolls stupid is going to
make them happy and not blinded with rage? Do they think it won't
provoke more insidious ways of board disruption?

Language is developed in real time and the only rigor necessitated by
it is hard wired neurologically and does not require a filter. So, in
lieu of developing filters, why don't you sit back and cry about it
and stop trying to censor human expression based on how intelligent
YOU say it is.
Useful, but misnamed.   (01/02/08 - 7:45 - by wizodd)
O.K. People, these postings are pretty good proof that it would be
useful.

As to infuriating stupid posters--who cares? They will be unable to
post and no one will know!

As to the rest of your questions about sarcasm, legal language, etc.

This is a TOOL folks. Like any tool, how you use it is up to you.

What is probably bothering you is that, while you read the title of
the article, you neglected to find out the goal of the program.

This is badly named, it is NOT a "stupidity filter" it is a language
structureal checker.

"Effective textual communication requires at least some formal rigor,
and we feel such rigor is worth encouraging and, at times,
enforcing."

Given that our universities are full of freshmen who cannot write a
complete thought in a paragraph--much less even a short paper on a
topic.

That if asked to do so, they call home to Mommy & Daddy on the phone
glued to their head and whine.

I'd say it's past due.

A FF add-in?
Sad   (04/14/08 - 9:59 - by StephZoR)
I wish this will be able to prevent people to comment about things
they don't know anything about. Those bastards.
That's almost what it will do, for example, no more "FAAAKE D8"
comments on UFO videos without solid arguments. That's cool!
If this is not intentionally a joke, it’s still unintentionally so.   (10/31/08 - 19:58 - by Swed Simon)
The name may be misguiding, but the concept is still moronic in
itself. The idea that you can reject a comments content as
self-evidentially useless solely because of ‘’gross prose style’’ is
too one-sided to be practical. A person can seriously lack grammatical
skills and still make valid points, and a comment can be within a
context where more emotionally rather than formally constructed
language is desired. Far from all internet conversation, even by
die-hard internet pseudo-intellectuals on a self-image trip, seek to
strictly live up to all these grammatical rules or even be smart to
begin with.
Continuation of last post   (10/31/08 - 20:01 - by Swed Simon)
Effective textual communication really does not requires this type of
formal rigor unless you have beyond average difficulties understanding
written text or got an uncontrollable urge to reject/feel superiority
over things that aren’t ‘’perfect’’. Even on their website they’ve
used relaxed pronunciation. Since their idea of filtering useless
comments is measuring lack of grammatical formality, why not count
relaxed pronunciation as stupid as well?

Oh well. Since being elitist for the sake of it is ‘’sort of the whole
point’’ I guess this whole project doesn’t really has to make much
sense. The creators and people actually using this can have fun with
the self-image fulfillment, though.

Add a New Comment
Your name:   0/20
Subject:  0/30
Your Comment:  0/999
Type the following letters: Visual CAPTCHA
Please keep your comments related to the above item's topic. TFOT reserves the right to delete any unrelated comment without notice.

Picture Of The Day
World's First Space Airport
World's First Space Airport

Video
Life of a Nematode
Life of a Nematode

Site Of The Week
Biology in Motion
Biology in Motion

Personal Column
Detroit’s Dream of Aircraft Production
Dr. Daniel Uziel
Detroit’s Dream of Aircraft Production
Terms Of Use | Privacy Policy | Contact Us | Advertise With Us | Site Profile
Copyright © 2007 The Future of Things. All rights reserved.