Ben Kurtovic
a73f618e0a
Initial conversion to Python 3
il y a 3 ans
Ben Kurtovic
9d66ebc6b2
copyvios: Config-directed URL proxying
il y a 3 ans
Ben Kurtovic
a463c6d052
Fix lazy loading bug where lxml.etree wasn't accessible to bs4.
il y a 7 ans
Ben Kurtovic
fbb9ea7b03
Catch empty Google results properly.
il y a 8 ans
Ben Kurtovic
aba91c0f1c
Missing comma.
il y a 8 ans
Ben Kurtovic
a95356676b
Add GoogleSearchEngine.
il y a 8 ans
Ben Kurtovic
98d0977c19
Refactor search; cleanup; fixup.
il y a 8 ans
Ben Kurtovic
76b068c4df
Add Yandex proxy support.
il y a 8 ans
Ben Kurtovic
a0d7eb62a2
Add Yandex search support.
il y a 8 ans
Ben Kurtovic
04ed5257c7
Refactor search engines.
il y a 8 ans
Ben Kurtovic
80890fb191
WebFileType doesn't work
il y a 8 ans
Ben Kurtovic
977b587e5e
Add support for Bing Search
il y a 8 ans
Ben Kurtovic
4e8be871b7
Update copyright year for 2015.
il y a 9 ans
Ben Kurtovic
b8d55973c9
Tell Yahoo! it's okay to return PDFs.
il y a 9 ans
Ben Kurtovic
693cdc302f
Catch errors while searching.
il y a 9 ans
Ben Kurtovic
4f5a22a2e5
Apparently oauth2 converts the query to unicode.
il y a 9 ans
Ben Kurtovic
5003c21ff6
Quoting the entire query works now.
il y a 9 ans
Ben Kurtovic
5677664476
Properly encode URL for the search engine.
il y a 9 ans
Ben Kurtovic
5890ee6e6a
Don't quote_plus() the query.
il y a 9 ans
Ben Kurtovic
5874467ec3
Bugfix, cleanup.
il y a 9 ans
Ben Kurtovic
3e4dac967d
Remove auto-quotes from queries; add min_query; halve max_query.
il y a 9 ans
Ben Kurtovic
39d5c7c149
Update copyright notices for 2014.
il y a 10 ans
Ben Kurtovic
ed95c99f0e
Update email address.
il y a 10 ans
Ben Kurtovic
0b7a13eca5
Update copyright notices for 2013.
il y a 11 ans
Ben Kurtovic
a4dda89a61
Various fixes for copyvios.
- Fix a bug in ExclusionsDB; improve URL regexes.
- NLTK's LookupError is actually an IOError.
- Fix bug in __repr__ for CopyvioCheckResult.
- Rewrite YahooBOSSSearchEngine to actually work with oauth2.
- Search engines now take a URL opener in addition to credentials.
il y a 11 ans
Ben Kurtovic
0ca84ab9bc
Implement lazy-importing of oauth2, nltk, and bs4.
il y a 11 ans
Ben Kurtovic
4baab6f57c
Implement lazy importing of root-level modules and packages.
- Simplify all imports
- Update dependency version in setup.py
- Change waitTime default from three seconds to two
il y a 11 ans
Ben Kurtovic
439b855254
Fully implement logging; fix non-unicode log messages.
il y a 12 ans
Ben Kurtovic
569c815d99
Implement NLTK for chunking article content ( #5 ).
il y a 12 ans
Ben Kurtovic
1af4217b63
Update copyright notices and some other improvements.
il y a 12 ans
Ben Kurtovic
d45e342bac
DOCUMENT EVERYTHING ( #5 )
Also implementing MWParserFromHell, plus some cleanup.
il y a 12 ans
Ben Kurtovic
d87c226417
__repr__ and __str__ for everything per #5 and #22 .
il y a 12 ans
Ben Kurtovic
7dbbe9683c
Update imports and exceptions.
il y a 12 ans
Ben Kurtovic
5ca1d91f3e
Use __all__ within e.w.copyvios and shorter imports
il y a 12 ans
Ben Kurtovic
d4e947b98b
earwigbot.wiki.copyvios.search module split
il y a 12 ans