A copyright violation detector running on Wikimedia Cloud Services https://tools.wmflabs.org/copyvios/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
Ben Kurtovic 190356b444 Update sites before doing check. 10 vuotta sitten
copyvios Finally fix #3; speed up highlighter with a deque. 10 vuotta sitten
logs Always have a log dir. 10 vuotta sitten
static Adjust indent size; fix rendering of strings. 10 vuotta sitten
templates Note that PDF parsing is now supported. 10 vuotta sitten
.gitignore Fix gitignore for logs. 10 vuotta sitten
.lighttpd.conf Begin conversion to Flask; updates. 10 vuotta sitten
LICENSE Updates. 10 vuotta sitten
README.md Link to some technical details. 10 vuotta sitten
app.fcgi Update sites before doing check. 10 vuotta sitten
build.py Begin conversion to Flask; updates. 10 vuotta sitten

README.md

This is a copyright violation detector running on Wikimedia Labs.

It can search the web for content similar to a given article, and graphically compare an article to a specific URL. Some technical details are expanded upon in a blog post.

Dependencies

Running

  • Install all dependencies listed above. You might want to use a virtualenv.

  • Create an SQL database with the cache and cache_data tables defined by earwigbot-plugins.

  • Create an earwigbot instance in .earwigbot (run earwigbot .earwigbot). In .earwigbot/config.yml, fill out the connection info for the database by adding the following to the wiki section:

      _copyviosSQL:
          host: <hostname of database server>
          db:   <name of database>
    

    If additional arguments are needed by oursql.connect(), like usernames or passwords, they should be added to the _copyviosSQL section.

  • Copy .lighttpd.conf to the relevant location (on Tool Labs, this is in the root of the project’s home directory) and adjust its contents as necessary.

  • Run ./build.py to minify JS and CSS files.

  • Adjust the hashbang in app.fcgi to point to the correct Python interpreter or virtual environment.

  • Start lighttpd (on Tool Labs, webservice start).