Timeline for Stack Overflow now uses machine learning to flag spam automatically
Current License: CC BY-SA 4.0
Post Revisions
20 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 27 at 16:47 | history | edited | SpevacusStaffMod | CC BY-SA 4.0 |
Properly differentiate the update from the original question
|
| Jan 21 at 18:22 | comment | added | Spevacus StaffMod | @chivracq Yep! We scan all posts from Staging Ground as well as all opinion based posts. I will also mention that SmokeDetector does monitor SG posts, it just might miss the ones you've seen. Or, it scanned them but did not auto flag them or find them to be spammy. | |
| Jan 21 at 18:17 | comment | added | chivracq | One "stupid" question, => is your Module also able to scan and auto-flag Posts (Questions - and Comments) from the 'SO'-'SG' ('Staging Ground')...? Because afaik, 'SD' is not, I regularly report in 'Charcoal' Posts from the 'SG' where after 1 hour, I am still the first and only Spam-flagger... (in the night hours European time...) | |
| Jan 21 at 18:03 | history | edited | SpevacusStaffMod | CC BY-SA 4.0 |
We're live!
|
| Jan 15 at 16:43 | history | edited | SpevacusStaffMod |
edited tags
|
|
| Jan 15 at 9:25 | comment | added | cafce25 | @starball I was hoping for an official answer to avoid subtleties like are the dates inclusive and stuff. You also need to subtract the "# Total Spam". I get "# Non-Spam" 8262 and thus a fp rate of ~0.08% from sede. | |
| Jan 15 at 9:06 | comment | added | cafce25 | Can we get the number of non-spam posts posted during the monitored timeframe so we can calculate the false-positive rate. | |
| Jan 15 at 2:08 | comment | added | chivracq | Next project using ML will be to create a Module detecting Duplicates, I hope...!?, to display the results just before an Asker will press the 'Submit' button, then again in the 'Staging Ground' and again if/once published on 'Main' when some Answerer starts typing in the 'Answer' field to post an Answer... (And I'll be absolutely extaticly super happy with "only" 64% detection, just saying...!!!) | |
| Jan 15 at 1:55 | comment | added | Spevacus StaffMod | @chivracq Remember that the perfect world we want here is to remove these autonomously and never have to involve a human. Like Dharman said, while we programmatically considered those 7 posts as false positives, a human review suggests these are largely posts we'd want deleted anyway. By my review, 5 of these are actual spam, and indeed one of them was deleted with spam flags after I made this post ( stackoverflow.com/staging-ground/79860571 ). The accuracy is likely far better than this post suggests, I was simply handing you the data I queried for with our internal data explorer. | |
| Jan 15 at 1:48 | comment | added | chivracq | I'm not that super impressed to be honest, 64% detection is definitely not that impressive, 94% is OK-ish, but the 7 binding false positives is unacceptable, don't fire any binding Flags at all, 'Smoke Detector' (and flagging Users) will still nuke the Post in less than 1 minute if it's indeed really Spam (except in the 'Staging Ground' where it can take up to 1 hour and often requires manual reporting in the 'SD/Charcoal' Chat-Room - due to lack of API-access, I think I understood/remember)... | |
| Jan 15 at 0:38 | answer | added | Starship | timeline score: 9 | |
| Jan 15 at 0:12 | comment | added | Dharman Mod | @Spevacus Out of the 7 you listed, I think only one is actually a false positive. | |
| Jan 14 at 22:14 | answer | added | starballMod | timeline score: 30 | |
| Jan 14 at 21:15 | history | became hot meta post | |||
| Jan 14 at 21:15 | comment | added | Spevacus StaffMod | @starball We can turn this on with just non-binding flags, but given the accuracy I'm personally fine with letting it loose. I'd really like to see how it does with no guardrails given the data we're looking at currently. However, if this is a precaution the community wants to take, I will bring it to the team. | |
| Jan 14 at 21:10 | history | edited | SpevacusStaffMod | CC BY-SA 4.0 |
After an audit, one of the false positives was actually deleted as spam, yay!
|
| Jan 14 at 21:10 | comment | added | Spevacus StaffMod | @starball Re: list - Sure! Theoretical false positive binding flag posts: stackoverflow.com/q/79850883 stackoverflow.com/q/79853510 stackoverflow.com/q/79854179 stackoverflow.com/q/79856691 stackoverflow.com/q/79856910 stackoverflow.com/q/79860571 stackoverflow.com/q/79862146 - An audit reveals one of the 8 posts was later deleted as spam, so we're actually at 7 false positives (updated). Some of these, upon review, are actually spam, but programmatically we've marked these as "misses" because they were, as an example, author-deleted | |
| Jan 14 at 20:59 | comment | added | starball Mod | nice! could you include a list of those 7 posts upon which false positive binding flags were cast? is there an option to try it out with binding flags disabled? | |
| Jan 14 at 20:21 | history | edited | SpevacusStaffMod | CC BY-SA 4.0 |
update screenshot for post notice
|
| Jan 14 at 20:13 | history | asked | SpevacusStaffMod | CC BY-SA 4.0 |