Ben Kurtovic
23663afbe2
Fix background URL generation
pirms 2 gadiem
Ben Kurtovic
0a81acfa5c
Even better error handling
pirms 3 gadiem
Ben Kurtovic
b134ef67ba
Even better error handling
pirms 3 gadiem
Ben Kurtovic
c829590bc9
Bugfix
pirms 3 gadiem
Ben Kurtovic
ad87596e37
Better error handling for revision IDs
pirms 3 gadiem
Ben Kurtovic
eacb294735
Update URLs
pirms 4 gadiem
Ben Kurtovic
4cebe36eee
Validate site info if not in sites DB
pirms 5 gadiem
Ben Kurtovic
96ac14ec38
Use rvslots for fetching revision content
pirms 5 gadiem
Ben Kurtovic
c73179b28c
Update highlighter for new MarkovChain implementation
pirms 5 gadiem
Ben Kurtovic
01cd6dc763
Make caching advisory and reduce db lock duration
pirms 5 gadiem
Ben Kurtovic
8321a07ba9
More additions for sqlite support
pirms 5 gadiem
Ben Kurtovic
ca58aac413
Add sqlite3 backend to replace mysql
pirms 5 gadiem
Ben Kurtovic
29861c1118
Revert previous; literal_eval is necessary here. Add error handling.
pirms 6 gadiem
Ben Kurtovic
a63e4e2236
Safer parsing for Turnitin API result.
pirms 6 gadiem
Ben Kurtovic
79866da842
Strip whitespace.
pirms 7 gadiem
Ben Kurtovic
088dfe6197
Again, fix link and cleanup.
pirms 7 gadiem
Ben Kurtovic
21aad51375
Fix unnecessary extra work.
pirms 7 gadiem
Ben Kurtovic
63f1ae98c4
Show warning when a page has an attribution template.
pirms 7 gadiem
Ben Kurtovic
a2df4bba0a
Tweak parameters.
pirms 7 gadiem
Ben Kurtovic
1076959f71
Bugfix in highlight_delta() call; clarify docs.
pirms 8 gadiem
Ben Kurtovic
2a81217de8
Add support for detailed text comparison in API (T132949)
pirms 8 gadiem
Ben Kurtovic
8be5c63bc1
Turnitin bugfix.
pirms 8 gadiem
Ben Kurtovic
137c33bfcb
Fix _format_date() for caches older than a day.
pirms 8 gadiem
Ben Kurtovic
e6e83ed73c
Revert "ast.literal_eval -> json.loads"
This reverts commit 6d940b5c92
pirms 8 gadiem
Ben Kurtovic
0692227496
Cleanup/tweaks, mainly to rendering.
pirms 8 gadiem
Ben Kurtovic
6d940b5c92
ast.literal_eval -> json.loads
pirms 8 gadiem
Ben Kurtovic
83f5588354
Disable turnitin by default for now, until we confirm it works as expected.
pirms 8 gadiem
4e994f1302
Refactor turnitin.py, incorporate diff link/timestamp
* Add a wiki timestamp parser to copyvios/misc.py
* Refactor copyvios/turnitin.py for more sensible structure
* Update templates/index.mako to incorporate diff link/timestamp and
make it clearer that Turnitin is revision-based checking
pirms 8 gadiem
1ffa87da0b
Improve turnitin.py docstrings, fix bugs
pirms 8 gadiem
bf0aa22fa9
[WIP] improve docstrings and naming, mark TODO
pirms 8 gadiem
efe2300250
[WIP] Basic working integration of turnitin
Add a checkbox to allow searching the EranBot/plagiabot database for
Turnitin results, and display them in a similar form to the on-wiki
EranBot reports if they exist.
Add a new module (copyvios/turnitin.py) to handle fetching and parsing
the EranBot results.
Bug: https://phabricator.wikimedia.org/T110144
TODO: tweak display HTML/CSS; refactor/clean up turnitin.py;
improve dev set-up so it doesn't always default to testwiki
and can test without hardcoding page title
pirms 8 gadiem
Ben Kurtovic
ad033a6e2b
Pass excluded URL data to the API.
pirms 8 gadiem
Ben Kurtovic
321e6e3352
Support new 'excluded' column in database.
pirms 8 gadiem
Ben Kurtovic
4092cbc0f6
Keep all workers' backgrounds in sync, and ensure they update at 00:00 UTC.
pirms 8 gadiem
Ben Kurtovic
b325ef84cd
Support general/update notices.
pirms 8 gadiem
Ben Kurtovic
1327f57404
Fix backgrounds when the image has been moved.
pirms 9 gadiem
Ben Kurtovic
f2c1d5b853
Make cache.langs, cache.projects sorted lists.
pirms 9 gadiem
Ben Kurtovic
0f4ecfe80a
Auto-bypass cache when cache_possible_miss is True.
pirms 9 gadiem
Ben Kurtovic
81552f28d3
Bugfix; missed column.
pirms 9 gadiem
Ben Kurtovic
10e75c7b7a
Cache 'possible_miss' result value.
pirms 9 gadiem
Ben Kurtovic
331b05baae
Truncate URLs above 1024 chars.
pirms 9 gadiem
Ben Kurtovic
a07d12699a
Finally fix #3 ; speed up highlighter with a deque.
pirms 9 gadiem
Ben Kurtovic
f4ff4099be
Fix old reference to query.all_projects.
pirms 9 gadiem
Ben Kurtovic
a1d8b79265
Fix recursion bug.
pirms 9 gadiem
Ben Kurtovic
6eed95c761
Use a global cache instead of the request-specific one.
pirms 9 gadiem
Ben Kurtovic
ddcf2a8963
Store what was previously in SQL tables in flask.g.
pirms 9 gadiem
Ben Kurtovic
9b07a5e9dc
OrderedDict requires extra parens.
pirms 9 gadiem
Ben Kurtovic
340ba07c9d
Generate API results as ordered dicts.
pirms 9 gadiem
Ben Kurtovic
6d87ad41ed
Rearrange functions in api.py.
pirms 9 gadiem
Ben Kurtovic
2fad6b0b60
Description of API response data.
pirms 9 gadiem
Frances Hocutt