i have 2 questions hope can answered semi-dependent on work. below grammar + tree grammar + java test file.
what trying achieve following:
question 1:
i have grammar parses language correctly. semantic checks on variable declarations. created tree walker , far semi works. problem it's not capturing whole string of expression. example,
float x = 10 + 10; it capturing first part, i.e. 10. not sure doing wrong. if did in 1 pass, works. somehow, if split work grammar , tree grammar, not capturing whole string.
question 2:
i check on rule such if conditions returns true, remove subtree. example,
float x = 10; float x; // <================ removed. i have tried using rewrite rules think more complex that.
test.g:
grammar test; options { language = java; output = ast; } parse : vardeclare+ ; vardeclare : type id equalexp? ';' ; equalexp : ('=' (expression | '...')) ; expression : binaryexpression ; binaryexpression : addingexpression (('=='|'!='|'<='|'>='|'>'|'<') addingexpression)* ; addingexpression : multiplyingexpression (('+'|'-') multiplyingexpression)* ; multiplyingexpression : unaryexpression (('*'|'/') unaryexpression)* ; unaryexpression: ('!'|'-')* primitiveelement; primitiveelement : literalexpression | id | '(' expression ')' ; literalexpression : int ; id : identifier ; type : 'int' | 'float' ; // l e x c l r u l e s int : digits ; identifier : letter (letter | digit)*; ws : ( ' ' | '\t' | '\r' | '\n' ) {$channel=hidden;} ; fragment letter : ('a'..'z' | 'a'..'z' | '_') ; fragment digits: digit+; fragment digit : '0'..'9'; testtree.g:
tree grammar testtree; options { language = java; tokenvocab = test; astlabeltype = commontree; } @members { semanticcheck s; public testtree(treenodestream input, semanticcheck s) { this(input); this.s = s; } } parse[semanticcheck s] : vardeclare+ ; vardeclare : type id equalexp? ';' {s.check($type.name, $id.text, $equalexp.expr);} ; equalexp returns [string expr] : ('=' (expression {$expr = $expression.e;} | '...' {$expr = "...";})) ; expression returns [string e] @after {$e = $expression.text;} : binaryexpression ; binaryexpression : addingexpression (('=='|'!='|'<='|'>='|'>'|'<') addingexpression)* ; addingexpression : multiplyingexpression (('+'|'-') multiplyingexpression)* ; multiplyingexpression : unaryexpression (('*'|'/') unaryexpression)* ; unaryexpression: ('!'|'-')* primitiveelement; primitiveelement : literalexpression | id | '(' expression ')' ; literalexpression : int ; id : identifier ; type returns [string name] @after { $name = $type.text; } : 'int' | 'float' ; java test file, test.java:
import java.util.arraylist; import java.util.list; import org.antlr.runtime.antlrstringstream; import org.antlr.runtime.commontokenstream; import org.antlr.runtime.rulereturnscope; import org.antlr.runtime.tree.commontree; import org.antlr.runtime.tree.commontreenodestream; public class test { public static void main(string[] args) throws exception { semanticcheck s = new semanticcheck(); string src = "float x = 10+y; \n" + "float x; \n"; testlexer lexer = new testlexer(new antlrstringstream(src)); //testlexer lexer = new testlexer(new antlrfilestream("input.txt")); commontokenstream tokenstream = new commontokenstream(lexer); testparser parser = new testparser(tokenstream); rulereturnscope r = parser.parse(); system.out.println("parse tree:\n" + tokenstream.tostring()); commontree t = (commontree)r.gettree(); commontreenodestream nodes = new commontreenodestream(t); nodes.settokenstream(tokenstream); testtree walker = new testtree(nodes, s); walker.parse(s); } } class semanticcheck { list<string> names; public semanticcheck() { this.names = new arraylist<string>(); } public boolean check(string type, string variablename, string exp) { system.out.println("type: " + type + " variablename: " + variablename + " exp: " + exp); if(names.contains(variablename)) { system.out.println("remove statement! defined!"); return true; } names.add(variablename); return false; } } thanks in advance!
i figured out problem , turns out needed build ast first before can anything. in understanding flat tree vs building ast.
how output ast built using antlr?
thanks bart's endless examples here in stackoverflow, able semantic predicates needed in example above.
below updated code:
test.g
grammar test; options { language = java; output = ast; } tokens { vardecl; assign = '='; eqt = '=='; neq = '!='; lt = '<'; lteq = '<='; gt = '>'; gteq = '>='; not = '!'; plus = '+'; minus = '-'; mult = '*'; div = '/'; } parse : vardeclare+ ; vardeclare : type id equalexp ';' -> ^(vardecl type id equalexp) ; equalexp : (assign^ (expression | '...' )) ; expression : binaryexpression ; binaryexpression : addingexpression ((eqt|neq|lteq|gteq|lt|gt)^ addingexpression)* ; addingexpression : multiplyingexpression ((plus|minus)^ multiplyingexpression)* ; multiplyingexpression : unaryexpression ((mult|div)^ unaryexpression)* ; unaryexpression: ((not|minus))^ primitiveelement | primitiveelement ; primitiveelement : literalexpression | id | '(' expression ')' -> expression ; literalexpression : int ; id : identifier ; type : 'int' | 'float' ; // l e x c l r u l e s int : digits ; identifier : letter (letter | digit)*; ws : ( ' ' | '\t' | '\r' | '\n' ) {$channel=hidden;} ; fragment letter : ('a'..'z' | 'a'..'z' | '_') ; fragment digits: digit+; fragment digit : '0'..'9'; this should automatically build ast whenever have vardeclare. on tree grammar/walker.
testtree.g
tree grammar testtree; options { language = java; tokenvocab = test; astlabeltype = commontree; output = ast; } tokens { removed; } @members { semanticcheck s; public testtree(treenodestream input, semanticcheck s) { this(input); this.s = s; } } start[semanticcheck s] : vardeclare+ ; vardeclare : ^(vardecl type id equalexp) -> {s.check($type.text, $id.text, $equalexp.text)}? removed -> ^(vardecl type id equalexp) ; equalexp : ^(assign expression) | ^(assign '...') ; expression : ^(('!') expression) | ^(('+'|'-'|'*'|'/') expression expression*) | ^(('=='|'<='|'<'|'>='|'>'|'!=') expression expression*) | literalexpression ; literalexpression : int | id ; id : identifier ; type : 'int' | 'float' ; now on test it:
test.java
import java.util.arraylist; import java.util.list; import org.antlr.runtime.antlrstringstream; import org.antlr.runtime.commontokenstream; import org.antlr.runtime.tree.*; public class test { public static void main(string[] args) throws exception { semanticcheck s = new semanticcheck(); string src = "float x = 10; \n" + "int x = 1; \n"; testlexer lexer = new testlexer(new antlrstringstream(src)); commontokenstream tokenstream = new commontokenstream(lexer); testparser parser = new testparser(tokenstream); testparser.parse_return r = parser.parse(); system.out.println("tree:" + ((tree)r.tree).tostringtree() + "\n"); commontreenodestream nodes = new commontreenodestream((tree)r.tree); nodes.settokenstream(tokenstream); testtree walker = new testtree(nodes, s); testtree.start_return r2 = walker.start(s); system.out.println("\ntree walker: "+((tree)r2.tree).tostringtree()); } } class semanticcheck { list<string> names; public semanticcheck() { this.names = new arraylist<string>(); } public boolean check(string type, string variablename, string exp) { system.out.println("type: " + type + " variablename: " + variablename + " exp: " + exp); if(names.contains(variablename)) { return true; } names.add(variablename); return false; } } output:
tree:(vardecl float x (= 10)) (vardecl int x (= 1)) type: float variablename: x exp: = 10 type: int variablename: x exp: = 1 tree walker: (vardecl float x (= 10)) removed hope helps! please feel free point errors if did wrong.
Comments
Post a Comment