ben
/
earwigbot
peilaus alkaen https://github.com/earwig/earwigbot
1
1
Fork 0
Koodi Julkaisut 3 Toiminta
993 Commitit
2 Branchit
4.4 MiB
Puu: 4e8be871b7
Branchit Tagit
${ item.name }
Create branch ${ searchTerm }
from '4e8be871b7'
${ noResults }
Commit Graph

38 Commitit (4e8be871b7d57e5512854240baefd1d3bf4268be)

Tekijä SHA1 Viesti Päivämäärä
  Ben Kurtovic 4e8be871b7 Update copyright year for 2015. 9 vuotta sitten
  Ben Kurtovic 9ffc3f1bf5 Raise file crawl size limit for PDFs. 9 vuotta sitten
  Ben Kurtovic 901192ec18 Handle errors from UnicodeDamnit. 9 vuotta sitten
  Ben Kurtovic 699f6e3b17 Seems it will sometimes raise AssertionError. 9 vuotta sitten
  Ben Kurtovic 12c5170815 Catch another exception thrown by pdfminer. 9 vuotta sitten
  Ben Kurtovic 77514ee925 Add another PDF string substitution. 9 vuotta sitten
  Ben Kurtovic 0bdcbca8b0 Rudimentary solution for PDF parsing (closes earwig/copyvios#18) 9 vuotta sitten
  Ben Kurtovic 30f72df470 Refactor parsers; fix empty document behavior. 9 vuotta sitten
  Ben Kurtovic 5349179088 Fix parsing of plain text documents (earwig/copyvios#3) 9 vuotta sitten
  Ben Kurtovic 7afb484cea Refactor a bunch of copyvio internals. Store all sources with a result object. 9 vuotta sitten
  Ben Kurtovic 54ddff049f Make CopyvioSource public; tweaks. 9 vuotta sitten
  Ben Kurtovic 9b87e2e5f7 Fix trying to remove a node that was already removed. 9 vuotta sitten
  Ben Kurtovic 193f96451e Also strip <ref>s in ArticleTextParser.strip(). 9 vuotta sitten
  Ben Kurtovic c4dede1459 Reorder length check to potentially fix an empty-query bug. 9 vuotta sitten
  Ben Kurtovic 5874467ec3 Bugfix, cleanup. 9 vuotta sitten
  Ben Kurtovic ae0c390ceb Redesign copyvio internals to parallelize URL loading/parsing. 9 vuotta sitten
  Ben Kurtovic 3e4dac967d Remove auto-quotes from queries; add min_query; halve max_query. 9 vuotta sitten
  Ben Kurtovic 6b146a397a Also strip out files and categories in ATP.strip(). 9 vuotta sitten
  Ben Kurtovic 2dfdf1bd4a Ensure the text is stripped properly. 9 vuotta sitten
  Ben Kurtovic 3dde1c5d60 Correctly handle HTML with no <body> tags. 10 vuotta sitten
  Ben Kurtovic 39d5c7c149 Update copyright notices for 2014. 10 vuotta sitten
  Ben Kurtovic ed95c99f0e Update email address. 10 vuotta sitten
  Ben Kurtovic 0b7a13eca5 Update copyright notices for 2013. 11 vuotta sitten
  Ben Kurtovic 8862bec3d9 Fix statements assigned to nothing. 11 vuotta sitten
  Ben Kurtovic a4dda89a61 Various fixes for copyvios. 11 vuotta sitten
  Ben Kurtovic 0ca84ab9bc Implement lazy-importing of oauth2, nltk, and bs4. 11 vuotta sitten
  Ben Kurtovic 4baab6f57c Implement lazy importing of root-level modules and packages. 11 vuotta sitten
  Ben Kurtovic 7d52d4558a Some updates to !lag. 11 vuotta sitten
  Ben Kurtovic 33aa1d6744 Collapse extra newlines to avoid distorting trigrams. 11 vuotta sitten
  Ben Kurtovic 5ab736723b Fixing a number of silly mistakes; refactoring (thanks pylint) 12 vuotta sitten
  Ben Kurtovic c260648bdb Finish chunking algorithm, improve !link, other fixes. 12 vuotta sitten
  Ben Kurtovic 569c815d99 Implement NLTK for chunking article content (#5). 12 vuotta sitten
  Ben Kurtovic cb87004107 Primitive screen scraper for HTML using BeautifulSoup and LXML. 12 vuotta sitten
  Ben Kurtovic 1af4217b63 Update copyright notices and some other improvements. 12 vuotta sitten
  Ben Kurtovic d45e342bac DOCUMENT EVERYTHING (#5) 12 vuotta sitten
  Ben Kurtovic d87c226417 __repr__ and __str__ for everything per #5 and #22. 12 vuotta sitten
  Ben Kurtovic 5ca1d91f3e Use __all__ within e.w.copyvios and shorter imports 12 vuotta sitten
  Ben Kurtovic 86a8440730 Moving parsers to own file. 12 vuotta sitten