7337638b72
Change form get_lexer to guess_lexer -- guess_lexer is more accurate.
10 years ago
Ben Kurtovic
7b9e98bc5b
Thanks @riamse
10 years ago
ca4bb8ff42
Change format of symbols to fit databse model.
10 years ago
Ben Kurtovic
b2d9ad5c97
Missed a line.
10 years ago
Ben Kurtovic
b6fabc54ec
Can't use executemany() here.
10 years ago
21cf52ea65
Call start_parse_servers from crawl.py
10 years ago
c9520fd4e3
Add function to start parse servers in parser/__init__.py
10 years ago
Ben Kurtovic
e64c81f66f
Consistency fix: Use Codelet.url instead of Codelet.code_url.
10 years ago
65a2688be9
Adjust test for new language dict.
10 years ago
Ben Kurtovic
f02dc4497c
Fixes.
10 years ago
Ben Kurtovic
5d8ac664fe
HAHA WHAT ARE ITERATORS
10 years ago
5e6e3fcbf5
Adjust test for new language dict.
10 years ago
94953624c8
Fix #34 .
Add:
bitshift/crawler/indexer.py
-Add a `try-except` block to catch the `UnsupportedFileError` exception.
10 years ago
Ben Kurtovic
5a83720617
Strip encoding lines.
10 years ago
fc8d478060
Untested fix #33 .
Add:
bitshift/crawler/indexer.py
-Add conditional to remove the full path of a repository if the owner's
directory contains only one sub-directory.
10 years ago
Ben Kurtovic
a3eacc287e
Try to make exception reporting more useful.
10 years ago
Ben Kurtovic
5d9ef2774d
Some fixes, mainly involving language detection.
10 years ago
Ben Kurtovic
9f935bbb74
This is ugly, but it improves the current setup.
10 years ago
b698a16c98
Add parse() and insert() calls to crawler.
Add:
bitshift/crawler/indexer.py
-Add `parse()` and `insert()` calls to `_insert_repository_codelets()`.
10 years ago
f8436fa484
Part of #26 . Move __init__.py to crawl.py.
Add:
bitshift/crawler/(__init__, crawl).py
-Move `__init__.py` to `crawl.py`, and add a `main` block to allow
running the crawler via `python -m`.
10 years ago
Ben Kurtovic
4dc83e913a
Implement /search.json route ( #27 )
10 years ago
Ben Kurtovic
52c7e9cf5e
Finish JSON stringification ( closes #29 )
10 years ago
Ben Kurtovic
326be3050f
Start work on serializing; clean up languages.
10 years ago
Ben Kurtovic
878088f9ab
Split symbols into implicit OR groups ( closes #28 )
10 years ago
Ben Kurtovic
860260d0ad
Finish Database.search() ( closes #19 )
10 years ago
Ben Kurtovic
1fbe0e823b
Implement most of _get_codelets_from_ids().
10 years ago
Ben Kurtovic
827ea09085
Only use dynamic ranks if the conditional is complex.
10 years ago
Ben Kurtovic
7309ca5532
Some fixes and adjustments; note.
10 years ago
Ben Kurtovic
ba8bde7689
Cleanup, fixes, additions.
10 years ago
Ben Kurtovic
fbc5eebad4
Refactor query production; fixes.
10 years ago
Ben Kurtovic
91256b1384
Finish parameterize() for Text.
10 years ago
Ben Kurtovic
f49ad1d281
Move parameterization functions to Node classes.
10 years ago
Ben Kurtovic
8fbfd4c45c
Fix _parse_node; _parse_term quote handling; should probably refactor.
10 years ago
Ben Kurtovic
08ce46faeb
Implement FTS for authors.
10 years ago
Ben Kurtovic
d3f6f226f1
Finish exploding symbols.
10 years ago
Ben Kurtovic
1111afc198
Explode dates and symbols.
10 years ago
Ben Kurtovic
cd27777f83
Start exploding query trees.
10 years ago
Ben Kurtovic
4fa8b9f444
Improve exception behavior; cleanup.
10 years ago
43c6b2ca01
Add languages.json, integrate with languages.py.
Add:
bitshift/resources
-Add directory for bitshift's Python resources.
bitshift/resources/languages.json
-Add `json` file containing all the programming languages supported by
bitshift.
bitshift/languages.py
-Set `LANGS` by reading `languages.json`.
10 years ago
Ben Kurtovic
7e876c835f
Add sort keys, fix a bug, add tests.
10 years ago
Ben Kurtovic
30406483b9
Add unit tests, cleanup.
10 years ago
Ben Kurtovic
983f9a5d94
Implement basic tree balancing.
10 years ago
Ben Kurtovic
c9b558d0b3
Finish query parsing, mostly.
10 years ago
Ben Kurtovic
611a4ea9b2
More accurate model for boolean logic.
10 years ago
Ben Kurtovic
d319bde7db
Begin work on a model for boolean logic parsing.
10 years ago
Ben Kurtovic
bcab13f027
Finish _scan_query(), _split_query().
10 years ago
Ben Kurtovic
da8759372a
Clean up SPLIT_QUERY_STRING.
10 years ago
Ben Kurtovic
9c867b4dc3
code
10 years ago
Ben Kurtovic
4100107e93
Change UnaryOp.NOT to work with BinaryOp.{OR,AND}
10 years ago
Ben Kurtovic
525b49b4b2
Construct full trees; support inverting with not:<term>.
10 years ago
Benjamin Attal
Severyn Kozak