 Ben Kurtovic
|
12247dd756
|
Add no_links and no_searches to copyvio_check().
|
pirms 9 gadiem |
|
Ben Kurtovic
|
f37621e5ec
|
Use a deque for a FIFO instead of the python list LIFO.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
8e439e1eea
|
source.join() now blocks when in the middle of processing.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
dbb1ae5483
|
Handle empty queues correctly. Remove some log messages.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
2fa8aeba5b
|
Fix a blocking issue.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
c56838e742
|
Only spawn one worker for comparisons in local mode.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
939d8be08f
|
Fix variable.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
3ed8837a3e
|
Fix stopping queues in local mode.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
de7576728f
|
Fix dequeueing logic a bit.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
b939262b11
|
Bugfix.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
32ef0fbf1f
|
Add a bunch of temporary debugging code.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
c7b3b7bc7f
|
CopyvioSource.workspace should be public.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
e73e626994
|
Some locks needed to be tightened.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
486c4692ed
|
Remove _workers attr of workspaces.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
7c0e98596c
|
Some bugfixes.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
361f7709f8
|
Starting work on global workers.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
bdcbfa5327
|
Catch errors around response.read().
|
pirms 9 gadiem |
|
Ben Kurtovic
|
9b87e2e5f7
|
Fix trying to remove a node that was already removed.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
24dd497fd9
|
Catch more general socket.error.
|
pirms 9 gadiem |
|
Ben Kurtovic
|
5e72e74759
|
Employ new piecewise article-delta confidence function.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
193f96451e
|
Also strip <ref>s in ArticleTextParser.strip().
|
pirms 10 gadiem |
|
Ben Kurtovic
|
c4dede1459
|
Reorder length check to potentially fix an empty-query bug.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
203c65280c
|
Float delta.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
6b0f8ad311
|
Fix reference.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
e2d7c7aef6
|
Update with new confidence function; fix unicode.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
05010933c7
|
Reorder some URL opening code; zip protection.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
4f5a22a2e5
|
Apparently oauth2 converts the query to unicode.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
5003c21ff6
|
Quoting the entire query works now.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
5677664476
|
Properly encode URL for the search engine.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
5890ee6e6a
|
Don't quote_plus() the query.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
2bddf79a3d
|
Fix deadlock when calling queue.put() while holding the mutex.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
7a4fcd7807
|
Fix queue clear call.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
efae85a1fe
|
Move thread spawning code to worker class.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
6a90efc812
|
Improve !threads command output.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
7137dda920
|
Update copyvio checker to not make concurrent requests to a single domain.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
5874467ec3
|
Bugfix, cleanup.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
a68bebc43c
|
Tweak git version form a bit.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
96631e25f4
|
Make lazy importing code thread-safe.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
cc7ac52a05
|
Fix query counting.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
d672e670fa
|
Fix param name.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
0e28f89466
|
Update logging.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
ae0c390ceb
|
Redesign copyvio internals to parallelize URL loading/parsing.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
c38fe56ad4
|
Fix edit counter link.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
3a8349e8ab
|
Allow regexes in exclusion lists.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
3e4dac967d
|
Remove auto-quotes from queries; add min_query; halve max_query.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
1501341000
|
Allow even more time for a URL to time out.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
6b146a397a
|
Also strip out files and categories in ATP.strip().
|
pirms 10 gadiem |
|
Ben Kurtovic
|
ccb3c022ca
|
Some servers don't leave a space before the content type parameter list.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
5e9d4cfa78
|
copyvios: use a different timeout for direct URL comparisons.
|
pirms 10 gadiem |
|
Ben Kurtovic
|
a0bc8d69ab
|
Merge pull request #50 from justinkim/fix/typo
Fix a typo
|
pirms 10 gadiem |
