Skip to content

Spam detection in Junction #273

Open

Description

@kracekumar

Recently @pythonhacker and @harisibrahimkv reported few comments looks like a spam. Junction right now only allows logged in users to comment.

There are two levels of deducting/marking comment as a spam. During signup check the reputation of email/ip of user, if it is potential spam email id/ip, block the user, legit user can write back to the concerned person to activate the account.

One more way is to have a spam deduction based on the content. There are two ways to implement, one user can report this as spam, once threshold is crossed, differentiate it in UI and automatic spam detector can mark this as spam.

Once user has crossed the threshold of spam posting, account should be blocked.

Reddit also does similar stuff.

@vigneshsarma thoughts ?

Activity

  1. pythonhacker commented on Jun 9, 2015

    @pythonhacker

    How about also,

    1. Thresholding comments from same login ? If you posted a comment on a talk at time t , you cannot post another comment within t + delta_t where delta_t could be even an hour.
    2. Not allowing postings from known spam IPs - there are well known lists we can scrub an internal SPAM IP list to.
    3. Not allowing postings with less than n characters. This prevents postings like All the best Sir!, Wonderful Sir! etc. This forces people to think and write something. Possibly keep n at something like 120 or 140 chars.
  2. kracekumar commented on Jun 10, 2015

    @kracekumar
    MemberAuthor

    All these three should be implemented.

  3. kracekumar commented on Jun 10, 2015

    @kracekumar
    MemberAuthor

    The order for implementing above should Point 3 -> Point 2 -> Point 1. We need to prevent spam account creation as well.

  4. theskumar commented on Jun 15, 2015

    @theskumar
    Member

    How about just use a third party to evaluate the spammy nature of a particular comment. I suggest using askimet[1] if anyone doesn't have any objection with it.

    I'm personally not in favour to including the spam detection logic coded into junction itself. If not askimet, then a publish pypi library that will give spam score for a comment should do.

    [1]
    https://github.com/miracle2k/python-akismet
    https://akismet.com/

  5. ChillarAnand commented on Jun 15, 2015

    @ChillarAnand
    Member

    akismet 👍

  6. kracekumar commented on Jul 5, 2015

    @kracekumar
    MemberAuthor

    Reading the docs suggests wordpress api key is required. I am sure, there will be other libraries/services, if some one can do quick prototype, it will be awesome.

  7. pythonhacker commented on Apr 6, 2017

    @pythonhacker

    I am taking this up. Will try and get it ready for PyCon 2017.

  8. self-assigned this
    on Apr 6, 2017
  9. pythonhacker commented on Apr 6, 2017

    @pythonhacker

    First will collect comments across last 3-4 PyCons and using one of the libraries - get its spam detection rating - on each. Idea is to find out how much true and false positives are reported. Will update here with some libraries which are pure Python as much as possible.

  10. changed the title [-]Spam deduction in Junction[/-] [+]Spam detection in Junction[/+] on Apr 6, 2017
  11. kracekumar commented on Apr 9, 2017

    @kracekumar
    MemberAuthor
  12. kracekumar commented on Jun 22, 2017

    @kracekumar
    MemberAuthor

    Here is a csv file which has spam and non-spam content from junction.

  13. added this to the Enhancements milestone on Mar 20, 2020
  14. ananyo2012 commented on Mar 20, 2020

    @ananyo2012
    Contributor

    Unfortunately the spam data is gone from one of the recent changes. Need to dump it somewhere, maybe in a different branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions