Ben Kurtovic
a73f618e0a
Initial conversion to Python 3
il y a 3 ans
Ben Kurtovic
9d66ebc6b2
copyvios: Config-directed URL proxying
il y a 3 ans
Ben Kurtovic
91846ce4fb
Refactor out mirror hinting logic in source parsers.
il y a 8 ans
Ben Kurtovic
03910b6cb5
Add mirror detection logic to parsers; fixes.
il y a 8 ans
Ben Kurtovic
bb819c9306
Explicitly include excluded URLs in the result set; mark as excluded.
il y a 8 ans
Ben Kurtovic
4e8be871b7
Update copyright year for 2015.
il y a 9 ans
Ben Kurtovic
5194525a32
Note when sources might have been missed.
il y a 9 ans
Ben Kurtovic
30f72df470
Refactor parsers; fix empty document behavior.
il y a 9 ans
Ben Kurtovic
f8f4669460
Remove unnecessary key attribute of sources.
il y a 9 ans
Ben Kurtovic
9fd145da5c
Add some docs; better sorting function.
il y a 9 ans
Ben Kurtovic
7afb484cea
Refactor a bunch of copyvio internals. Store all sources with a result object.
il y a 9 ans
Ben Kurtovic
54ddff049f
Make CopyvioSource public; tweaks.
il y a 9 ans
Ben Kurtovic
ae0c390ceb
Redesign copyvio internals to parallelize URL loading/parsing.
il y a 10 ans
Ben Kurtovic
39d5c7c149
Update copyright notices for 2014.
il y a 10 ans
Ben Kurtovic
ed95c99f0e
Update email address.
il y a 10 ans
Ben Kurtovic
0b7a13eca5
Update copyright notices for 2013.
il y a 11 ans
Ben Kurtovic
bcf9b70107
Keep track of how long generating results takes; support 'max_time'.
il y a 11 ans
Ben Kurtovic
a4dda89a61
Various fixes for copyvios.
- Fix a bug in ExclusionsDB; improve URL regexes.
- NLTK's LookupError is actually an IOError.
- Fix bug in __repr__ for CopyvioCheckResult.
- Rewrite YahooBOSSSearchEngine to actually work with oauth2.
- Search engines now take a URL opener in addition to credentials.
il y a 11 ans
Ben Kurtovic
a074da853b
More work on copyvios, including an exclusions database ( #5 )
* Added exclusions module with a fully implemented ExclusionsDB that can pull
from multiple sources for different sites.
* Moved CopyvioCheckResult to its own module, to be imported by __init__.
* Some other related changes.
il y a 12 ans