Ogre::Compiler2Pass Class Reference

Compiler2Pass is a generic 2 pass compiler/assembler. More...

#include <OgreCompiler2Pass.h>

Inheritance diagram for Ogre::Compiler2Pass:

List of all members.


Public Member Functions
	Compiler2Pass ()
	constructor
virtual	~Compiler2Pass ()
bool	compile (const String &source, const String &sourceName)
	compile the source - performs 2 passes.
virtual const String &	getClientBNFGrammer (void)=0
	gets BNF Grammer.
virtual const String &	getClientGrammerName (void)=0
	get the name of the BNF grammer.
Protected Types
typedef std::vector< TokenRule >	TokenRuleContainer
typedef TokenRuleContainer::iterator	TokenRuleIterator
typedef std::vector< LexemeTokenDef >	LexemeTokenDefContainer
typedef LexemeTokenDefContainer::iterator	LexemeTokenDefIterator
typedef std::map< std::string, size_t >	LexemeTokenMap
typedef LexemeTokenMap::iterator	TokenKeyIterator
typedef std::vector< TokenInst >	TokenInstContainer
typedef TokenInstContainer::iterator	TokenInstIterator
enum	OperationType { otUNKNOWN, otRULE, otAND, otOR, otOPTIONAL, otREPEAT, otDATA, otNOT_TEST, otEND }
enum	SystemRuleToken { _no_token_ = SystemTokenBase, _character_, _value_, _no_space_skip_ }
enum	BNF_ID { BNF_UNKOWN = 0, BNF_SYNTAX, BNF_RULE, BNF_IDENTIFIER, BNF_IDENTIFIER_RIGHT, BNF_IDENTIFIER_CHARACTERS, BNF_ID_BEGIN, BNF_ID_END, BNF_CONSTANT_BEGIN, BNF_SET_RULE, BNF_EXPRESSION, BNF_AND_TERM, BNF_OR_TERM, BNF_TERM, BNF_TERM_ID, BNF_CONSTANT, BNF_OR, BNF_TERMINAL_SYMBOL, BNF_TERMINAL_START, BNF_REPEAT_EXPRESSION, BNF_REPEAT_BEGIN, BNF_REPEAT_END, BNF_SET, BNF_SET_BEGIN, BNF_SET_END, BNF_NOT_TEST, BNF_NOT_TEST_BEGIN, BNF_OPTIONAL_EXPRESSION, BNF_NOT_EXPRESSION, BNF_NOT_CHK, BNF_OPTIONAL_BEGIN, BNF_OPTIONAL_END, BNF_NO_TOKEN_START, BNF_SINGLEQUOTE, BNF_SINGLE_QUOTE_EXC, BNF_SET_END_EXC, BNF_ANY_CHARACTER, BNF_SPECIAL_CHARACTERS1, BNF_SPECIAL_CHARACTERS2, BNF_WHITE_SPACE_CHK, BNF_LETTER, BNF_LETTER_DIGIT, BNF_DIGIT, BNF_WHITE_SPACE, BNF_ALPHA_SET, BNF_NUMBER_SET, BNF_SPECIAL_CHARACTER_SET1, BNF_SPECIAL_CHARACTER_SET2, BNF_SPECIAL_CHARACTER_SET3, BNF_NOT_CHARS }
Protected Member Functions
bool	doPass1 ()
	perform pass 1 of compile process scans source for lexemes that can be tokenized and then performs general semantic and context verification on each lexeme before it is tokenized.
bool	doPass2 ()
	performs Pass 2 of compile process which is execution of the tokens
virtual void	executeTokenAction (const size_t tokenID)=0
	execute the action associated with the token pointed to by the Pass 2 token instruction position.
virtual void	setupTokenDefinitions (void)=0
	setup client token definitions.
const TokenInst &	getNextToken (const size_t expectedTokenID=0)
	Gets the next token from the instruction que.
const TokenInst &	getCurrentToken (void)
	Gets the current token from the instruction que.
bool	testNextTokenID (const size_t expectedTokenID)
	If a next token instruction exist then test if its token ID matches.
void	replaceToken (void)
float	getNextTokenValue (void)
	Gets the next token's associated floating point value in the instruction que that was parsed from the text source.
const String &	getNextTokenLabel (void)
	Gets the next token's associated text label in the instruction que that was parsed from the text source.
size_t	getPass2TokenQueCount (void) const
	Gets the number of tokens waiting in the instruction que that need to be processed by an token action in pass 2.
size_t	getRemainingTokensForAction (void) const
	Get the number of tokens not processed by action token.
void	addLexemeToken (const String &lexeme, const size_t token, const bool hasAction=false, const bool caseSensitive=false)
	Add a lexeme token association.
void	setClientBNFGrammer (void)
	sets up the parser rules for the client based on the BNF Grammer text passed in.
void	findEOL ()
	find the eol charater
bool	isFloatValue (float &fvalue, size_t &charsize) const
	check to see if the text at the present position in the source is a numerical constant
bool	isCharacterLabel (const size_t rulepathIDX)
	Check if source at current position is supposed to be a user defined character label.
bool	isLexemeMatch (const String &lexeme, const bool caseSensitive) const
	check to see if the text is in the lexeme text library
bool	positionToNextLexeme ()
	position to the next possible valid sysmbol
bool	processRulePath (size_t rulepathIDX)
	process input source text using rulepath to determine allowed tokens
void	setActiveContexts (const uint contexts)
	setup ActiveContexts - should be called by subclass to setup initial language contexts
void	skipComments ()
	comment specifiers are hard coded
void	skipEOL ()
	find end of line marker and move past it
void	skipWhiteSpace ()
	skip all the white space which includes spaces and tabs
bool	ValidateToken (const size_t rulepathIDX, const size_t activeRuleID)
	check if current position in source has the lexeme text equivalent to the TokenID
void	verifyTokenRuleLinks (const String &grammerName)
	scan through all the rules and initialize token definition with index to rules for non-terminal tokens.
void	checkTokenActionTrigger (void)
	Checks the last token instruction and if it has an action then it triggers the action of the previously found token having an action.
String	getBNFGrammerTextFromRulePath (size_t ruleID)
	Get the text representation of the rule path.
Protected Attributes
TokenState *	mClientTokenState
TokenState *	mActiveTokenState
	Active token que, definitions, rules currntly being used by parser.
size_t	mPass2TokenQuePosition
	the location within the token instruction container where pass 2 is
size_t	mPreviousActionQuePosition
	the que position of the previous token that had an action.
const String *	mSource
	pointer to the source to be compiled
String	mSourceName
	name of the source to be compiled
size_t	mEndOfSource
size_t	mCurrentLine
size_t	mCharPos
	current line number in source being tokenized
std::map< size_t, float >	mConstants
	storage container for constants defined in source container uses Token index as a key associated with a float constant
std::map< size_t, String >	mLabels
	storage container for string labels defined in source container uses Token index as a key associated with a label
bool	mLabelIsActive
	flag indicates when a label is being parsed.
size_t	mActiveLabelKey
	the key of the active label being built during pass 1.
bool	mNoSpaceSkip
	flag being true indicates that spaces are not to be skipped automatically gets set to false when mLabelIsActive goes to false
bool	mNoTerminalToken
	if flag is true then next terminal token is not added to token que if found but does effect rule path flow
uint	mActiveContexts
	Active Contexts pattern used in pass 1 to determine which tokens are valid for a certain context.
Static Protected Attributes
const size_t	SystemTokenBase = 1000
Private Types
typedef std::map< String, TokenState >	TokenStateContainer
Private Member Functions
void	activatePreviousTokenAction (void)
	if a previous token action was setup then activate it now
void	initBNFCompiler (void)
	initialize token definitions and rule paths
void	buildClientBNFRulePaths (void)
	Convert BNF grammer token que created in pass 1 into a BNF rule path.
void	modifyLastRule (const OperationType pendingRuleOp, const size_t tokenID)
	modify the last rule in the container. An end operation is added to the rule path.
size_t	getClientLexemeTokenID (const String &lexeme, const bool isCaseSensitive=false)
	get the token ID for a lexeme in the client state.
void	extractNonTerminal (const OperationType pendingRuleOp)
	Extract a Non Terminal identifier from the token que.
void	extractTerminal (const OperationType pendingRuleOp, const bool notoken=false)
	Extract a Terminal lexeme from the token que and add to current rule expression.
void	extractSet (const OperationType pendingRuleOp)
	Extract a set from the token que and add to current rule expression.
void	extractNumericConstant (const OperationType pendingRuleOp)
	Extract a numeric constant from the token que and add it to the current rule expression.
String	getLexemeText (size_t &ruleID)
Static Private Attributes
TokenState	mBNFTokenState
TokenStateContainer	mClientTokenStates

Detailed Description

Compiler2Pass is a generic 2 pass compiler/assembler.

Remarks:: provides a tokenizer in pass 1 and relies on the subclass to provide the virtual method for pass 2

PASS 1 - tokenize source: this is a simple brute force lexical scanner/analyzer that also parses the formed token for proper semantics and context in one pass it uses Look Ahead Left-Right (LALR) ruling based on Backus - Naur Form (BNF) notation for semantic checking and also performs context checking allowing for language dialects

PASS 2 - generate application specific instructions ie native instructions based on the tokens in the instruction container.

: this class must be subclassed with the subclass providing some implementation details for Pass 2. The subclass is responsible for setting up the token libraries along with defining the language syntax and handling token actions during the second pass.

: The sub class normally supplies a simplified BNF text description in its constructor prior to doing any parsing/tokenizing of source. The simplified BNF text description defines the language syntax and rule structure. The meta-symbols used in the BNF text description are:

::= meaning "is defined as". "::=" starts the definition of a rule. The left side of ::= must contain an

<> angle brackets are used to surround syntax rule names. A syntax rule name is also called a non-terminal in that it does not generate a terminal token in the instruction container for pass 2 processing.

| meaning "or". if the item on the left of the | fails then the item on the right is tested. Example: ::= 'true' | 'false'; whitespace is used to imply AND operation between left and right items. Example: ::= 'terrain_shadows' the 'terrain_shadows' terminal token must be found and rule must pass in order for rule to pass.

[] optional rule identifier is enclosed in meta symbols [ and ]. Note that only one identifier or terminal token can take [] modifier. {} repetitive identifier (zero or more times) is enclosed in meta symbols { and } Note that only one identifier or terminal token can take {} modifier. '' terminal tokens are surrounded by single quotes. A terminal token is always one or more characters. For example: 'Colour' defines a character sequence that must be matched in whole. Note that matching is case sensitive. @ turn on single character scanning and don't skip white space. Mainly used for label processing that allow white space. Example: '@ ' prevents the white space between the quotes from being skipped -'' no terminal token is generated when a - precedes the first single quote but the text in between the quotes is still tested against the characters in the source being parsed. (?! ) negative lookahead (not test) inspired by Perl 5. Scans ahead for a non-terminal or terminal expression that should fail in order to make the rule production pass. Does not generate a token or advance the cursur. If the lookahead result fails ie token is found, then the current rule fails and rollback occurs. Mainly used to solve multiple contexts of a token. An Example of where not test is used to solve multiple contexts:

::= "::=" ::= { } ::= "|" ::= { } ::= | | | ::= (?!"::=")

appears on both sides of the ::= so (?!"::=") test to make sure that ::= is not on the right which would indicate that a new rule was being formed.

Works on both terminals and non-terminals. Note: lookahead failure cause the whole rule to fail and rollback to occur

<#name> # indicates that a numerical value is to be parsed to form a terminal token. Name is optional and is just a descriptor to help with understanding what the value will be used for. Example: ::= <#red> <#green> <#blue>

() parentheses enclose a set of characters that can be used to generate a user identifier. for example: (0123456789) matches a single character found in that set. An example of a user identifier:

::= {} ::= (abcdefghijklmnopqrstuvwxyz)

This will generate a rule that accepts one or more lowercase letters to make up the Label. The User identifier stops collecting the characters into a string when a match cannot be found in the rule.

Definition at line 114 of file OgreCompiler2Pass.h.

Member Typedef Documentation

typedef std::vector<LexemeTokenDef> Ogre::Compiler2Pass::LexemeTokenDefContainer [protected]

Definition at line 185 of file OgreCompiler2Pass.h.

typedef LexemeTokenDefContainer::iterator Ogre::Compiler2Pass::LexemeTokenDefIterator [protected]

Definition at line 186 of file OgreCompiler2Pass.h.

typedef std::map<std::string, size_t> Ogre::Compiler2Pass::LexemeTokenMap [protected]

Definition at line 188 of file OgreCompiler2Pass.h.

typedef std::vector<TokenInst> Ogre::Compiler2Pass::TokenInstContainer [protected]

Definition at line 203 of file OgreCompiler2Pass.h.

typedef TokenInstContainer::iterator Ogre::Compiler2Pass::TokenInstIterator [protected]

Definition at line 204 of file OgreCompiler2Pass.h.

typedef LexemeTokenMap::iterator Ogre::Compiler2Pass::TokenKeyIterator [protected]

Definition at line 189 of file OgreCompiler2Pass.h.

typedef std::vector<TokenRule> Ogre::Compiler2Pass::TokenRuleContainer [protected]

Definition at line 135 of file OgreCompiler2Pass.h.

typedef TokenRuleContainer::iterator Ogre::Compiler2Pass::TokenRuleIterator [protected]

Definition at line 136 of file OgreCompiler2Pass.h.

typedef std::map<String, TokenState> Ogre::Compiler2Pass::TokenStateContainer [private]

Definition at line 449 of file OgreCompiler2Pass.h.

Member Enumeration Documentation

enum Ogre::Compiler2Pass::BNF_ID [protected]

Enumeration values:

BNF_UNKOWN

BNF_SYNTAX

BNF_RULE

BNF_IDENTIFIER

BNF_IDENTIFIER_RIGHT

BNF_IDENTIFIER_CHARACTERS

BNF_ID_BEGIN

BNF_ID_END

BNF_CONSTANT_BEGIN

BNF_SET_RULE

BNF_EXPRESSION

BNF_AND_TERM

BNF_OR_TERM

BNF_TERM

BNF_TERM_ID

BNF_CONSTANT

BNF_OR

BNF_TERMINAL_SYMBOL

BNF_TERMINAL_START

BNF_REPEAT_EXPRESSION

BNF_REPEAT_BEGIN

BNF_REPEAT_END

BNF_SET

BNF_SET_BEGIN

BNF_SET_END

BNF_NOT_TEST

BNF_NOT_TEST_BEGIN

BNF_OPTIONAL_EXPRESSION

BNF_NOT_EXPRESSION

BNF_NOT_CHK

BNF_OPTIONAL_BEGIN

BNF_OPTIONAL_END

BNF_NO_TOKEN_START

BNF_SINGLEQUOTE

BNF_SINGLE_QUOTE_EXC

BNF_SET_END_EXC

BNF_ANY_CHARACTER

BNF_SPECIAL_CHARACTERS1

BNF_SPECIAL_CHARACTERS2

BNF_WHITE_SPACE_CHK

BNF_LETTER

BNF_LETTER_DIGIT

BNF_DIGIT

BNF_WHITE_SPACE

BNF_ALPHA_SET

BNF_NUMBER_SET

BNF_SPECIAL_CHARACTER_SET1

BNF_SPECIAL_CHARACTER_SET2

BNF_SPECIAL_CHARACTER_SET3

BNF_NOT_CHARS

Definition at line 146 of file OgreCompiler2Pass.h.

enum Ogre::Compiler2Pass::OperationType [protected]

Enumeration values:

otUNKNOWN

otRULE

otAND

otOR

otOPTIONAL

otREPEAT

otDATA

otNOT_TEST

otEND

Definition at line 120 of file OgreCompiler2Pass.h.

enum Ogre::Compiler2Pass::SystemRuleToken [protected]

Enumeration values:

_no_token_

_character_

_value_

_no_space_skip_

Definition at line 139 of file OgreCompiler2Pass.h.

Constructor & Destructor Documentation

Ogre::Compiler2Pass::Compiler2Pass ( )

constructor

virtual Ogre::Compiler2Pass::~Compiler2Pass ( ) [virtual]

Definition at line 478 of file OgreCompiler2Pass.h.

Member Function Documentation

void Ogre::Compiler2Pass::activatePreviousTokenAction ( void ) [private]

if a previous token action was setup then activate it now

void Ogre::Compiler2Pass::addLexemeToken ( const String & lexeme,

const size_t token,

const bool hasAction = false,

const bool caseSensitive = false

) [protected]

Add a lexeme token association.
The backend compiler uses the associations between lexeme and token when building the rule base from the BNF script so all associations must be done prior to compiling a source.
Parameters:

lexeme is the name of the token and use when parsing the source to determin a match for a token.

token is the ID associated with the lexeme

hasAction must be set true if the client wants an action triggered when this token is generated

caseSensitive should be set true if lexeme match should use case sensitivity

void Ogre::Compiler2Pass::buildClientBNFRulePaths ( void ) [private]

Convert BNF grammer token que created in pass 1 into a BNF rule path.

void Ogre::Compiler2Pass::checkTokenActionTrigger ( void ) [protected]

Checks the last token instruction and if it has an action then it triggers the action of the previously found token having an action.

bool Ogre::Compiler2Pass::compile ( const String & source,

const String & sourceName

)

compile the source - performs 2 passes.
First pass is to tokinize, check semantics and context. The second pass is performed by using tokens to look up function implementors and executing them which convert tokens to application specific instructions.
Remarks:
Pass 2 only gets executed if Pass 1 has built enough tokens to complete a rule path and found no errors

Parameters:

source a pointer to the source text to be compiled

Returns:
true if Pass 1 and Pass 2 are successfull false if any errors occur in Pass 1 or Pass 2

bool Ogre::Compiler2Pass::doPass1 ( ) [protected]

perform pass 1 of compile process scans source for lexemes that can be tokenized and then performs general semantic and context verification on each lexeme before it is tokenized.
A tokenized instruction list is built to be used by Pass 2. A rule path can trigger Pass 2 execution if enough tokens have been generated in Pass 1. Pass 1 will then pass control to pass 2 temporarily until the current tokens have been consumed.

bool Ogre::Compiler2Pass::doPass2 ( ) [protected]

performs Pass 2 of compile process which is execution of the tokens

Remarks:
Pass 2 takes the token instructions generated in Pass 1 and builds the application specific instructions along with verifying symantic and context rules that could not be checked in Pass 1.

Pass 2 execution consumes tokens and moves the Pass 2 token instruction position towards the end of the token container. Token execution can insert new tokens into the token container.

virtual void Ogre::Compiler2Pass::executeTokenAction ( const size_t tokenID ) [protected, pure virtual]

execute the action associated with the token pointed to by the Pass 2 token instruction position.
Its upto the child class to implement how it will associate a token key with and action. Actions should be placed at positions withing the BNF grammer (instruction que) that indicate enough tokens exist for pass 2 processing to take place.
Implemented in Ogre::CompositorScriptCompiler, and Ogre::MaterialScriptCompiler.

void Ogre::Compiler2Pass::extractNonTerminal ( const OperationType pendingRuleOp ) [private]

Extract a Non Terminal identifier from the token que.

void Ogre::Compiler2Pass::extractNumericConstant ( const OperationType pendingRuleOp ) [private]

Extract a numeric constant from the token que and add it to the current rule expression.

void Ogre::Compiler2Pass::extractSet ( const OperationType pendingRuleOp ) [private]

Extract a set from the token que and add to current rule expression.

void Ogre::Compiler2Pass::extractTerminal ( const OperationType pendingRuleOp,

const bool notoken = false

) [private]

Extract a Terminal lexeme from the token que and add to current rule expression.

void Ogre::Compiler2Pass::findEOL ( ) [protected]

find the eol charater

String Ogre::Compiler2Pass::getBNFGrammerTextFromRulePath ( size_t ruleID ) [protected]

Get the text representation of the rule path.
This is a good way to way to visually verify that the BNF grammer did compile correctly.
Parameters:

ruleID is the index into the rule path.

virtual const String& Ogre::Compiler2Pass::getClientBNFGrammer ( void ) [pure virtual]

gets BNF Grammer.
Gets called when BNF grammer has to be compiled for the first time.
Implemented in Ogre::CompositorScriptCompiler, and Ogre::MaterialScriptCompiler.

virtual const String& Ogre::Compiler2Pass::getClientGrammerName ( void ) [pure virtual]

get the name of the BNF grammer.

Implemented in Ogre::CompositorScriptCompiler, and Ogre::MaterialScriptCompiler.

size_t Ogre::Compiler2Pass::getClientLexemeTokenID ( const String & lexeme,

const bool isCaseSensitive = false

) [private]

get the token ID for a lexeme in the client state.
If the lexeme is not found then it is added to the map and definition container and a new tokenID created.
Returns:
the ID of the token.

const TokenInst& Ogre::Compiler2Pass::getCurrentToken ( void ) [protected]

Gets the current token from the instruction que.

String Ogre::Compiler2Pass::getLexemeText ( size_t & ruleID ) [private]

const TokenInst& Ogre::Compiler2Pass::getNextToken ( const size_t expectedTokenID = 0 ) [protected]

Gets the next token from the instruction que.
If an unkown token is found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

Parameters:

expectedTokenID if set then if tokenID does not match then an exception is raised

const String& Ogre::Compiler2Pass::getNextTokenLabel ( void ) [protected]

Gets the next token's associated text label in the instruction que that was parsed from the text source.
If an unkown token is found or no associated label was found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

float Ogre::Compiler2Pass::getNextTokenValue ( void ) [protected]

Gets the next token's associated floating point value in the instruction que that was parsed from the text source.
If an unkown token is found or no associated value was found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

size_t Ogre::Compiler2Pass::getPass2TokenQueCount ( void ) const [protected]

Gets the number of tokens waiting in the instruction que that need to be processed by an token action in pass 2.

size_t Ogre::Compiler2Pass::getRemainingTokensForAction ( void ) const [protected]

Get the number of tokens not processed by action token.
Client Actions should use this method to retreive the number of parameters(tokens) remaining to be processed in the action.

void Ogre::Compiler2Pass::initBNFCompiler ( void ) [private]

initialize token definitions and rule paths

bool Ogre::Compiler2Pass::isCharacterLabel ( const size_t rulepathIDX ) [protected]

Check if source at current position is supposed to be a user defined character label.
A new label is processed when previous operation was not _character_ otherwise the processed character (if match was found) is added to the current label. This allows _character_ operations to be chained together to form a crude regular expression to build a label.
Parameters:

rulepathIDX index into rule path database of token to validate.

Returns:
true if token was found for character label.

bool Ogre::Compiler2Pass::isFloatValue ( float & fvalue,

size_t & charsize

) const [protected]

check to see if the text at the present position in the source is a numerical constant

Parameters:

fvalue is a reference that will receive the float value that is in the source

charsize reference to receive number of characters that make of the value in the source

Returns:
true if characters form a valid float representation false if a number value could not be extracted

bool Ogre::Compiler2Pass::isLexemeMatch ( const String & lexeme,

const bool caseSensitive

) const [protected]

check to see if the text is in the lexeme text library

Parameters:

lexeme points to begining of text where a lexem token might exist

caseSensitive set to true if match should be case sensitive

Returns:
true if a matching token could be found in the token type library false if could not be tokenized

void Ogre::Compiler2Pass::modifyLastRule ( const OperationType pendingRuleOp,

const size_t tokenID

) [private]

modify the last rule in the container. An end operation is added to the rule path.

bool Ogre::Compiler2Pass::positionToNextLexeme ( ) [protected]

position to the next possible valid sysmbol

bool Ogre::Compiler2Pass::processRulePath ( size_t rulepathIDX ) [protected]

process input source text using rulepath to determine allowed tokens

Remarks:
the method is reentrant and recursive if a non-terminal token is encountered in the current rule path then the method is called using the new rule path referenced by the non-terminal token Tokens can have the following operation states which effects the flow path of the rule RULE: defines a rule path for the non-terminal token AND: the token is required for the rule to pass OR: if the previous tokens failed then try these ones OPTIONAL: the token is optional and does not cause the rule to fail if the token is not found REPEAT: the token is required but there can be more than one in a sequence DATA: Used by a previous token ie for character sets NOTTEST: performs negative lookahead ie make sure the next token is not of a certain type END: end of the rule path - the method returns the succuss of the rule

Parameters:

rulepathIDX index into an array of Token Rules that define a rule path to be processed

Returns:
true if rule passed - all required tokens found false if one or more tokens required to complete the rule were not found

void Ogre::Compiler2Pass::replaceToken ( void ) [protected]

void Ogre::Compiler2Pass::setActiveContexts ( const uint contexts ) [protected]

setup ActiveContexts - should be called by subclass to setup initial language contexts

Definition at line 405 of file OgreCompiler2Pass.h.
References Ogre::uint.

void Ogre::Compiler2Pass::setClientBNFGrammer ( void ) [protected]

sets up the parser rules for the client based on the BNF Grammer text passed in.
Raises an exception if the grammer did not compile successfully. This method gets called when a call to compile occurs and no compiled BNF grammer exists, otherwise nothing will happen since the compiler has no rules to work with. The grammer only needs to be set once during the lifetime of the compiler unless the grammer changes. BNF Grammer rules are cached once the BNF grammer source is compiled. The client should never have to call this method directly.

virtual void Ogre::Compiler2Pass::setupTokenDefinitions ( void ) [protected, pure virtual]

setup client token definitions.
Gets called when BNF grammer is being setup.
Implemented in Ogre::CompositorScriptCompiler, and Ogre::MaterialScriptCompiler.

void Ogre::Compiler2Pass::skipComments ( ) [protected]

comment specifiers are hard coded

void Ogre::Compiler2Pass::skipEOL ( ) [protected]

find end of line marker and move past it

void Ogre::Compiler2Pass::skipWhiteSpace ( ) [protected]

skip all the white space which includes spaces and tabs

bool Ogre::Compiler2Pass::testNextTokenID ( const size_t expectedTokenID ) [protected]

If a next token instruction exist then test if its token ID matches.
This method is usefull for peeking ahead during pass 2 to see if a certain token exists.
Parameters:

expectedTokenID is the ID of the token to match.

bool Ogre::Compiler2Pass::ValidateToken ( const size_t rulepathIDX,

const size_t activeRuleID

) [protected]

check if current position in source has the lexeme text equivalent to the TokenID

Parameters:

rulepathIDX index into rule path database of token to validate

activeRuleID index of non-terminal rule that generated the token

Returns:
true if token was found false if token lexeme text does not match the source text if token is non-terminal then processRulePath is called

void Ogre::Compiler2Pass::verifyTokenRuleLinks ( const String & grammerName ) [protected]

scan through all the rules and initialize token definition with index to rules for non-terminal tokens.
Gets called when internal grammer is being verified or after client grammer has been parsed.
Parameters:

grammerName is the name of the grammer the token rules are for

Member Data Documentation

uint Ogre::Compiler2Pass::mActiveContexts [protected]

Active Contexts pattern used in pass 1 to determine which tokens are valid for a certain context.

Definition at line 255 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mActiveLabelKey [protected]

the key of the active label being built during pass 1.
a new key is calculated when mLabelIsActive switches from false to true
Definition at line 246 of file OgreCompiler2Pass.h.

TokenState* Ogre::Compiler2Pass::mActiveTokenState [protected]

Active token que, definitions, rules currntly being used by parser.

Definition at line 218 of file OgreCompiler2Pass.h.

TokenState Ogre::Compiler2Pass::mBNFTokenState [static, private]

Definition at line 447 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mCharPos [protected]

current line number in source being tokenized

Definition at line 233 of file OgreCompiler2Pass.h.

TokenState* Ogre::Compiler2Pass::mClientTokenState [protected]

Definition at line 215 of file OgreCompiler2Pass.h.

TokenStateContainer Ogre::Compiler2Pass::mClientTokenStates [static, private]

Definition at line 450 of file OgreCompiler2Pass.h.

std::map<size_t, float> Ogre::Compiler2Pass::mConstants [protected]

storage container for constants defined in source container uses Token index as a key associated with a float constant

Definition at line 237 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mCurrentLine [protected]

Definition at line 232 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mEndOfSource [protected]

Definition at line 230 of file OgreCompiler2Pass.h.

bool Ogre::Compiler2Pass::mLabelIsActive [protected]

flag indicates when a label is being parsed.
It gets set false when a terminal token not of _character_ is encountered
Definition at line 243 of file OgreCompiler2Pass.h.

std::map<size_t, String> Ogre::Compiler2Pass::mLabels [protected]

storage container for string labels defined in source container uses Token index as a key associated with a label

Definition at line 240 of file OgreCompiler2Pass.h.

bool Ogre::Compiler2Pass::mNoSpaceSkip [protected]

flag being true indicates that spaces are not to be skipped automatically gets set to false when mLabelIsActive goes to false

Definition at line 249 of file OgreCompiler2Pass.h.

bool Ogre::Compiler2Pass::mNoTerminalToken [protected]

if flag is true then next terminal token is not added to token que if found but does effect rule path flow

Definition at line 252 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mPass2TokenQuePosition [protected]

the location within the token instruction container where pass 2 is

Definition at line 220 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::mPreviousActionQuePosition [protected]

the que position of the previous token that had an action.
A token's action is fired on the next token having an action.
Definition at line 224 of file OgreCompiler2Pass.h.

const String* Ogre::Compiler2Pass::mSource [protected]

pointer to the source to be compiled

Definition at line 227 of file OgreCompiler2Pass.h.

String Ogre::Compiler2Pass::mSourceName [protected]

name of the source to be compiled

Definition at line 229 of file OgreCompiler2Pass.h.

const size_t Ogre::Compiler2Pass::SystemTokenBase = 1000 [static, protected]

Definition at line 138 of file OgreCompiler2Pass.h.

The documentation for this class was generated from the following file:

OgreCompiler2Pass.h

Ogre::Compiler2Pass Class Reference

Public Member Functions

Protected Types

Protected Member Functions

Protected Attributes

Static Protected Attributes

Private Types

Private Member Functions

Static Private Attributes