Ben Kurtovic
ca58aac413
Add sqlite3 backend to replace mysql
pirms 5 gadiem
Ben Kurtovic
a2df4bba0a
Tweak parameters.
pirms 7 gadiem
Ben Kurtovic
8be5c63bc1
Turnitin bugfix.
pirms 8 gadiem
Ben Kurtovic
137c33bfcb
Fix _format_date() for caches older than a day.
pirms 8 gadiem
Ben Kurtovic
83f5588354
Disable turnitin by default for now, until we confirm it works as expected.
pirms 8 gadiem
Frances Hocutt
efe2300250
[WIP] Basic working integration of turnitin
Add a checkbox to allow searching the EranBot/plagiabot database for
Turnitin results, and display them in a similar form to the on-wiki
EranBot reports if they exist.
Add a new module (copyvios/turnitin.py) to handle fetching and parsing
the EranBot results.
Bug: https://phabricator.wikimedia.org/T110144
TODO: tweak display HTML/CSS; refactor/clean up turnitin.py;
improve dev set-up so it doesn't always default to testwiki
and can test without hardcoding page title
pirms 8 gadiem
Ben Kurtovic
321e6e3352
Support new 'excluded' column in database.
pirms 8 gadiem
Ben Kurtovic
0f4ecfe80a
Auto-bypass cache when cache_possible_miss is True.
pirms 9 gadiem
Ben Kurtovic
81552f28d3
Bugfix; missed column.
pirms 9 gadiem
Ben Kurtovic
10e75c7b7a
Cache 'possible_miss' result value.
pirms 9 gadiem
Ben Kurtovic
331b05baae
Truncate URLs above 1024 chars.
pirms 9 gadiem
Ben Kurtovic
ddcf2a8963
Store what was previously in SQL tables in flask.g.
pirms 9 gadiem
Ben Kurtovic
89feddba98
Some minor tweaks, plus documentation for API parameters.
pirms 9 gadiem
Ben Kurtovic
6639338fa8
First version of new API (for #7 )
pirms 9 gadiem
Ben Kurtovic
6f574e37a2
Pluralize cache times correctly.
pirms 9 gadiem
Ben Kurtovic
b3e1862726
Catch and display search errors more usefully.
pirms 9 gadiem
Ben Kurtovic
b7bafb29cf
Implement noskip.
pirms 9 gadiem
Ben Kurtovic
73315e3370
FML.
pirms 9 gadiem
Ben Kurtovic
5242d31a93
Whoops.
pirms 9 gadiem
Ben Kurtovic
0731eb089d
Cleanup; fix confidence value comparison; fix.
pirms 9 gadiem
Ben Kurtovic
7cd9a53209
Fix oursql coercing binary data to unicode.
pirms 9 gadiem
Ben Kurtovic
b2fba1fa56
Implement new caching system and result structure.
pirms 9 gadiem
Ben Kurtovic
c6b9c9ebe6
Really fix checkbox blank value treatment.
pirms 9 gadiem
Ben Kurtovic
6cd194f971
Catch empty method error; fix blank values in query.
pirms 9 gadiem
Ben Kurtovic
b90e5e7279
Fix logic.
pirms 9 gadiem
Ben Kurtovic
c165e97fb2
Handle empty URLs from cached results correctly.
pirms 9 gadiem
Ben Kurtovic
710447a6bf
Slightly modified interface; give more options ( closes #17 ).
pirms 9 gadiem
Ben Kurtovic
8f6a02a8f3
If no scheme is given for the URL, add http://
pirms 9 gadiem
Ben Kurtovic
8d80eccad0
T_POSSIBLE back down to 0.4.
pirms 9 gadiem
Ben Kurtovic
490022910f
Update T_POSSIBLE, T_SUSPECT.
pirms 9 gadiem
Ben Kurtovic
9e13ac1c04
Increase log message verbosity; fix import.
pirms 10 gadiem
Ben Kurtovic
ce96f0f8a6
Use a yellow 'possible' box for confidence between 0.3 and 0.6 ( closes #5 ).
pirms 10 gadiem
Ben Kurtovic
75b627cfa5
Refactor out page.copyvio_compare() and surrounding code.
pirms 10 gadiem
Ben Kurtovic
cf66804fe5
Give a different error message for timeouts. Cheating a bit.
pirms 10 gadiem
Ben Kurtovic
7d426a4eb0
Support following redirects ( closes #6 ).
pirms 10 gadiem
Ben Kurtovic
a08bd49a1d
Support searching by revision ID ( closes #8 ).
pirms 10 gadiem
Ben Kurtovic
ca09caa15d
Give an error when no text could be found inside a URL.
pirms 10 gadiem
Ben Kurtovic
9a25f97831
Connection pooling; cleanup.
pirms 10 gadiem
Ben Kurtovic
cea99098ce
Fix some URL stuff.
pirms 10 gadiem
Ben Kurtovic
d2dff9f05d
Fix bad URI error thing.
pirms 10 gadiem
Ben Kurtovic
cb79aa24b4
Update/fix index.mako.
pirms 10 gadiem
Ben Kurtovic
6045ea4e44
Updates.
pirms 10 gadiem
Ben Kurtovic
4c411e7d9c
Update/merge CSS, JS, images.
pirms 10 gadiem
Ben Kurtovic
8027179e60
Update more references.
pirms 10 gadiem
Ben Kurtovic
859f388ee3
Fix bug when page contains unicode.
pirms 11 gadiem
Ben Kurtovic
adf6e9c23a
Properly handle bad URI schemes.
pirms 11 gadiem
Ben Kurtovic
e62383731c
Another bugfix.
pirms 11 gadiem
Ben Kurtovic
ea1b56b1c1
Some fixes and cleanup.
pirms 11 gadiem
Ben Kurtovic
6bbd4e00d1
Push actual checker code to master.
pirms 11 gadiem
Ben Kurtovic
958ad39739
Processing time has been moved into EarwigBot main.
pirms 11 gadiem