Package translate :: Package misc :: Module sparse :: Class SimpleParser
[hide private]
[frames] | no frames]

Class SimpleParser

source code

this is a simple parser

Instance Methods [hide private]
 
__init__(self, defaulttokenlist=None, whitespacechars=' \t\r\n', includewhitespacetokens=0) source code
 
stringtokenize(self, text)
makes strings in text into tokens...
source code
 
keeptogether(self, text)
checks whether a token should be kept together
source code
 
isstringtoken(self, text)
checks whether a token is a string token
source code
 
separatetokens(self, text, tokenlist=None)
this separates out tokens in tokenlist from whitespace etc
source code
 
removewhitespace(self, text)
this removes whitespace but lets it separate things out into separate tokens
source code
 
applytokenizer(self, inputlist, tokenizer)
apply a tokenizer to a set of text, flattening the result
source code
 
applytokenizers(self, inputlist, tokenizers)
apply a set of tokenizers to a set of text, flattening each time
source code
 
tokenize(self, source, tokenizers=None)
tokenize the text string with the standard tokenizers
source code
 
findtokenpos(self, tokennum)
finds the position of the given token in the text
source code
 
getlinepos(self, tokenpos)
finds the line and character position of the given character
source code
 
raiseerror(self, message, tokennum)
raises a ParserError
source code