Benjamin Attal
3fd25b944e
Change parser commands into subprocesses rather than servers.
pirms 10 gadiem
Benjamin Attal
ed4bc75f1e
Update parser commands, change parsers to subprocesses rather than servers, implement ruby parser with subprocess.
pirms 10 gadiem
Benjamin Attal
54a3485837
Add parsing for import statements in java.
pirms 10 gadiem
Benjamin Attal
6659e1fe73
Add block position function to handle positions for java ast nodes.
pirms 10 gadiem
Benjamin Attal
554314c40a
Move ruby parser to Ripper.
pirms 10 gadiem
Benjamin Attal
12ad10bad1
Rename modules 'namespaces'
pirms 10 gadiem
Ben Kurtovic
8431390bb3
Don't print the symbols.
pirms 10 gadiem
Ben Kurtovic
a4c428f4ff
Java parser: use 'functions' instead of 'methods'.
pirms 10 gadiem
Ben Kurtovic
8f39ce2f59
Ruby: use 'namespaces' instead of 'modules'.
pirms 10 gadiem
Benjamin Attal
e295044ec2
Fix null row issue.
pirms 10 gadiem
Benjamin Attal
335027bf45
Remove null values for rows.
pirms 10 gadiem
Benjamin Attal
2d643b1069
Stop ruby parser from failing. Add other parser fixes. Should be good
to go now.
pirms 10 gadiem
Benjamin Attal
78665f28d3
Fix for parser
pirms 10 gadiem
Benjamin Attal
16775a6490
Remove negative line numbers for parsers.
pirms 10 gadiem
Benjamin Attal
966ef007d5
Debug parse init function.
pirms 10 gadiem
Benjamin Attal
e9f0ddfe71
Ensure ruby parser returns all dictionary keys, regardless of whether or not there are elements in them.
pirms 10 gadiem
Benjamin Attal
73dee778c5
Can specify which port to run ruby server on.
pirms 10 gadiem
Benjamin Attal
6edb142b4a
Can specify port number to run java server on.
pirms 10 gadiem
Benjamin Attal
1f56300c73
Make sure parse servers return json loadable data. Make sure Java server closes client connection.
pirms 10 gadiem
Benjamin Attal
5d3baf49d1
Change format of string returned from ruby server to correct symbol table format.
pirms 10 gadiem
Benjamin Attal
0fb2cacbd4
Make sure parse servers return json loadable data. Make sure Java server closes client connection.
pirms 10 gadiem
Benjamin Attal
0a7cccc74c
Change format of string returned from ruby server to correct symbol table format.
pirms 10 gadiem
Benjamin Attal
be7c871cc9
Add Rakefile task for running ruby parse server.
pirms 10 gadiem
Benjamin Attal
d127ac94ad
1) All unavailable line numbers and column numbers become -1.
2) Add correct dependency in pom.xml
pirms 10 gadiem
Benjamin Attal
044a448602
Change the format of the symbols to fit with earwig's issue.
pirms 10 gadiem
Benjamin Attal
d8048a74f0
Fix data length sent to client from ruby server. Pad with extra bytes.
pirms 10 gadiem
Benjamin Attal
b16bc40d3f
Consolidate parsers into __init__.py. Update python.py parser.
pirms 10 gadiem
Benjamin Attal
6e54eb5147
Java server tells python client how much data to read.
pirms 10 gadiem
Benjamin Attal
7f1d9dd2d3
Add a working preliminary version of the ruby parser. Still need to add
a rule for running it in the Rakefile.
Add:
parser_server.rb:
- listens for connections from the python client process
parser.rb:
- creates a syntax tree from the input and returns relevant data
about it to the client
pirms 10 gadiem
Benjamin Attal
08f16074fb
Add template for ruby parser
pirms 10 gadiem
Benjamin Attal
2d7c1f4768
Fix array out of bounds exception coming from JavaParser.java
pirms 10 gadiem
Benjamin Attal
64ef9b04f2
Remove unecessary imports
pirms 10 gadiem
Benjamin Attal
f451e426e0
Refactor of the Java Parser
Mod:
Parser.java:
- Moved client reading and writing methods to the abstract
parser class, so that it is not specific to the JavaParser
JavaParser.java:
- Implemented NodeVisitor._cache. The cache is a stack of data
packets. When a node that we want information on is first
visited, a new packet of data is pushed onto the stack. The
child nodes of that original node than add information to the
packet, and when the original node is traversed again on the
way up the tree, the data is popped from the cache and added
to the symbols. This makes it possible to gather information
about various levels of the tree easily.
JavaSymbols.java:
- Refactor all the insertMethods to simply add a packet of data
to the appropriate HashMap.
Symbols.java
- Add a createCoord method which returns an arraylist
representing a point in a document.
pirms 10 gadiem
Benjamin Attal
2338887a52
Working version of java parser up and running.
pirms 10 gadiem
Benjamin Attal
19a5457f07
Change director structure for java
pirms 10 gadiem
Benjamin Attal
306875dae7
Make Parser implement runnable so parsing tasks can be started in separate threads. Make Parser constructor accept a client socket, add reading and writing methods for the socket to JavaParser. Parse main method sets up a server for accepting parse jobs from the crawler, and starts threads for each parse task.
pirms 10 gadiem
Benjamin Attal
77e2b6f524
Fix errors in java parser, mostly casting issues. In Parse.java, set up a tcp server for communication with python processes. Builds with maven
pirms 10 gadiem
Benjamin Attal
669c30cac7
Mod:
Parse.java:
Added comments
JavaParser.java:
Updated the genSymbols method and a private class 'NodeVisitor' which
implements ASTVisitor. genSymbols returns an instance of the
Symbols class containing all relevant data about the Java code.
JavaSymbols.java:
Add fields which map class, interface, method, field, and
variable names to positions.
pirms 10 gadiem
Benjamin Attal
63b09caa6c
Changed directory structure of java parser. Decided on multiple parsers in different languages, refactored bitshift/parser to fit with that paradigm.
pirms 10 gadiem
Benjamin Attal
a1066dd093
Modify parser/__init__.py so that it communicates with the Java parsing process and reads a result back from a unique file. Add template files for Java parsers.
pirms 10 gadiem