Ben Kurtovic
461f0fe052
More complete and descriptive error messages.
пре 10 година
Ben Kurtovic
3fbe0c2559
Finish parsing for languages.
пре 10 година
Ben Kurtovic
064c448824
Symbol parsing works.
пре 10 година
Ben Kurtovic
ed6d0258b2
Fix a typo.
пре 10 година
Ben Kurtovic
21d1c49ead
Parsing dates should work.
пре 10 година
Ben Kurtovic
27ac48ea6c
More debug stuff.
пре 10 година
Ben Kurtovic
c9258f35e4
Literal parsing; term parsing hooks; unit test stubs.
пре 10 година
Ben Kurtovic
816d003dd4
More work on query parsing.
пре 10 година
Ben Kurtovic
7b54df6335
Merge branch 'feature/parser' into feature/query_parser
пре 10 година
Ben Kurtovic
b5c22d3b4a
More work.
пре 10 година
Ben Kurtovic
cf2b48e217
More work on query tree structure.
пре 10 година
Ben Kurtovic
674f227b22
Work more on query tree structure.
пре 10 година
Benjamin Attal
be7c871cc9
Add Rakefile task for running ruby parse server.
пре 10 година
Benjamin Attal
d127ac94ad
1) All unavailable line numbers and column numbers become -1.
2) Add correct dependency in pom.xml
пре 10 година
Benjamin Attal
044a448602
Change the format of the symbols to fit with earwig's issue.
пре 10 година
Benjamin Attal
4cc0626a71
Catch ClassNotFound error in parser __init__.py
пре 10 година
Benjamin Attal
d8048a74f0
Fix data length sent to client from ruby server. Pad with extra bytes.
пре 10 година
Benjamin Attal
b16bc40d3f
Consolidate parsers into __init__.py. Update python.py parser.
пре 10 година
Benjamin Attal
71dec1d269
Modify the python parser. Make data more descriptive by adding data
about function calls.
пре 10 година
Benjamin Attal
6e54eb5147
Java server tells python client how much data to read.
пре 10 година
Benjamin Attal
d8b234f462
Update docstrings and parser dispatching in parser init file.
пре 10 година
Benjamin Attal
7f1d9dd2d3
Add a working preliminary version of the ruby parser. Still need to add
a rule for running it in the Rakefile.
Add:
parser_server.rb:
- listens for connections from the python client process
parser.rb:
- creates a syntax tree from the input and returns relevant data
about it to the client
пре 10 година
Benjamin Attal
08f16074fb
Add template for ruby parser
пре 10 година
Benjamin Attal
c859416d2d
Change test file to support different parsers
пре 10 година
Benjamin Attal
2d7c1f4768
Fix array out of bounds exception coming from JavaParser.java
пре 10 година
Benjamin Attal
64ef9b04f2
Remove unecessary imports
пре 10 година
Benjamin Attal
f451e426e0
Refactor of the Java Parser
Mod:
Parser.java:
- Moved client reading and writing methods to the abstract
parser class, so that it is not specific to the JavaParser
JavaParser.java:
- Implemented NodeVisitor._cache. The cache is a stack of data
packets. When a node that we want information on is first
visited, a new packet of data is pushed onto the stack. The
child nodes of that original node than add information to the
packet, and when the original node is traversed again on the
way up the tree, the data is popped from the cache and added
to the symbols. This makes it possible to gather information
about various levels of the tree easily.
JavaSymbols.java:
- Refactor all the insertMethods to simply add a packet of data
to the appropriate HashMap.
Symbols.java
- Add a createCoord method which returns an arraylist
representing a point in a document.
пре 10 година
Benjamin Attal
2338887a52
Working version of java parser up and running.
пре 10 година
Benjamin Attal
19a5457f07
Change director structure for java
пре 10 година
Benjamin Attal
306875dae7
Make Parser implement runnable so parsing tasks can be started in separate threads. Make Parser constructor accept a client socket, add reading and writing methods for the socket to JavaParser. Parse main method sets up a server for accepting parse jobs from the crawler, and starts threads for each parse task.
пре 10 година
Benjamin Attal
77e2b6f524
Fix errors in java parser, mostly casting issues. In Parse.java, set up a tcp server for communication with python processes. Builds with maven
пре 10 година
Benjamin Attal
669c30cac7
Mod:
Parse.java:
Added comments
JavaParser.java:
Updated the genSymbols method and a private class 'NodeVisitor' which
implements ASTVisitor. genSymbols returns an instance of the
Symbols class containing all relevant data about the Java code.
JavaSymbols.java:
Add fields which map class, interface, method, field, and
variable names to positions.
пре 10 година
Benjamin Attal
63b09caa6c
Changed directory structure of java parser. Decided on multiple parsers in different languages, refactored bitshift/parser to fit with that paradigm.
пре 10 година
Benjamin Attal
a1066dd093
Modify parser/__init__.py so that it communicates with the Java parsing process and reads a result back from a unique file. Add template files for Java parsers.
пре 10 година
Benjamin Attal
3bc748242d
Refactor parser/__init__.py for new parsing mechanism
пре 10 година
Benjamin Attal
430b7d3588
Remove unecessary submodule.
пре 10 година
Benjamin Attal
a8f918f7c4
Update class names. Move language ids to languages.py
пре 10 година
Benjamin Attal
0a57cf50e6
Add first version of the c parser
Add:
c.py
- CTreeCutter class is very similar to PyTreeCutter. It utilizes
self.cache as opposed to PyTreeCutter which doesn't yet.
- CTreeCutter visit functions simply add start and end lines of
the node to the cache, and visit_Decl pushes the cache onto
accum.
- parse_c performs a task identical to parse_py. However, many
c files need to be pre-processed before they are parsed.
пре 10 година
Benjamin Attal
847410b13c
Minor fix-ups in python parser.
Mod:
python.py
- Add self.cache to allow for saving of unassocaited metadata as the
PyTreeCutter moves down the syntax tree.
- Update docstrings.
пре 10 година
Benjamin Attal
d485b87f21
Fix docstring in bitshift/parser/python.py
пре 10 година
Benjamin Attal
b77db873c1
Refactor parsing in python by adding node visitor class. Performs same
tasks as previous version, but is more concise.
Add:
bitshift/parser/python.py:
Add PyTreeCutter class to perform actions on specific nodes.
пре 10 година
Benjamin Attal
4d8c818c05
Corrected documentation in bitshift/codelet.py and
bitshift/parser/__init__.py
пре 10 година
Benjamin Attal
5db273a773
Bugfixes for _serialize function in bitshift/parser/python.py
пре 10 година
Benjamin Attal
0c5e4572f8
Add placeholder functions for parsing c and java in bitshift/parser. Add parse_py function with helper functions. Parse_py grabs relevant information on variables, functions, and classes from abstract syntax tree of codelet code.
пре 10 година
Benjamin Attal
903e4ccc05
Add constants in bitshift/config.py for languages instead of just strings.
пре 10 година
Benjamin Attal
efdcb3793a
Add docstrings for functions in parser. Add ivar for syntax tree to codelet documentation.
пре 10 година
Benjamin Attal
d88e68e16e
Add dispatch 'parse' function to parser __init__.py. Basic code language identification as well. Included pycparser as a depedency.
пре 10 година
Ben Kurtovic
4dfd297472
Update some documentation.
пре 10 година
Ben Kurtovic
c4816c2bb8
Merge branch 'develop' into feature/query_parser
пре 10 година
Ben Kurtovic
2cf98df3e2
Merge branch 'develop' of github.com:earwig/bitshift into develop
Conflicts:
app.py
setup.py
пре 10 година