|
Blender V5.3
|
#include <token_stream.hh>
Inherits blender::gpu::shader::parser::LexerBase.
Inherited by blender::gpu::shader::parser::Parser< ExpressionLexer, NullParser >, and blender::gpu::shader::parser::Parser< LexerFn, ParserFn >.
Public Member Functions | |
| Token | operator[] (int i) const |
| void | build_scope_tree (ErrorHandler &err_handler) |
| void | build_token_to_scope_map () |
| void | parse_bsl (ErrorHandler &err_handler) |
| ast::LocalScope | root () const |
| void | print_ast () const |
| Public Member Functions inherited from blender::gpu::shader::parser::LexerBase | |
| std::string_view | token_types_str () const |
| void | identify_keywords () |
| void | identify_template_tokens () |
| void | reset_template_tokens () |
| Public Member Functions inherited from lexit::TokenBuffer | |
| TokenBuffer ()=default | |
| TokenBuffer (const std::string_view str, const CharClass char_class_table[128]) | |
| void | process (const std::string_view str, const CharClass char_class_table[128]) |
| void | clear () |
| Discard all currently held data. Does not reallocate. | |
| void | reserve (const uint32_t count) |
| Allocate backing memory for the given number of tokens and move currently held data. | |
| void | tokenize (const CharClass char_class_table[128]) |
| Tokenizes the input string by grouping contiguous characters of the same class. | |
| void | merge_complex_literals (bool bsl_template_angle_bracket_lexing=false) |
| Merge complex literals such as floats, strings and comments. | |
| void | atomize_words (IdentifierMap &identifiers, const KeywordTable &keywords) |
| Assign keyword types and atoms for a small set of identifier. | |
| void | compute_lengths () |
| Compute small token length for speeding up certain tasks. | |
| uint32_t | size () const |
| Return the amount of token inside the buffer. | |
| std::string_view | substr (const Token &start, const Token &end, const bool with_trailing_whitespaces=false) const |
| Return the substring between the start and end tokens (included). | |
| void | append (TokenType type, TokenAtom atom, int32_t str_size, int32_t str_size_with_witespaces) |
| Append a Token at the end of the buffer. | |
| Token | operator[] (int index) const |
| TokenMut | operator[] (int index) |
| TokenIt | begin () |
| TokenIt | end () |
| Token | front () |
| Token | back () |
Public Attributes | |
| int | include_id = 0 |
| std::string_view | scope_types_str |
| std::vector< ScopeType > | scope_types |
| std::vector< IndexRange > | scope_ranges |
| std::vector< ScopeLinks > | scope_links |
| std::vector< int > | token_scope |
| std::vector< ast::NodeData > | ast_nodes |
| Public Attributes inherited from lexit::TokenBuffer | |
| std::string_view | str_ |
| AlignedArrayPtr< TokenType > | types_ |
| AlignedArrayPtr< uint32_t > | offsets_ |
| AlignedArrayPtr< uint32_t > | offsets_end_ |
| AlignedArrayPtr< uint8_t > | lengths_ |
| AlignedArrayPtr< TokenAtom > | atoms_ |
| uint32_t | size_ = 0 |
| uint32_t | allocated_size_ = 0 |
Additional Inherited Members | |
| Static Public Attributes inherited from blender::gpu::shader::parser::LexerBase | |
| static const std::array< CharClass, 128 > | bsl_char_class_table |
| static const std::array< CharClass, 128 > | default_char_class_table |
Create semantic scopes from token stream. Also creates mapping table from token to scope to have bi-directional mapping.
Definition at line 81 of file token_stream.hh.
| void blender::gpu::shader::parser::ParserBase::build_scope_tree | ( | ErrorHandler & | err_handler | ) |
Definition at line 1125 of file grammar.cc.
References blender::gpu::shader::parser::LexerBase::identify_template_tokens(), blender::gpu::shader::parser::LexerBase::reset_template_tokens(), scope_links, scope_ranges, scope_types, lexit::TokenBuffer::size(), and blender::gpu::shader::parser::ScopeParser::translation_unit().
| void blender::gpu::shader::parser::ParserBase::build_token_to_scope_map | ( | ) |
Definition at line 425 of file intermediate.cc.
References build_token_to_scope_map(), scope_ranges, lexit::TokenBuffer::size(), and token_scope.
Referenced by build_token_to_scope_map().
| Token blender::gpu::shader::parser::ParserBase::operator[] | ( | int | i | ) | const |
Definition at line 441 of file intermediate.cc.
References operator[](), and lexit::Token::Token().
Referenced by operator[]().
| void blender::gpu::shader::parser::ParserBase::parse_bsl | ( | ErrorHandler & | err_handler | ) |
Definition at line 1339 of file bsl/grammar.cc.
References ast_nodes, blender::gpu::shader::parser::LexerBase::identify_template_tokens(), bsl::BSLParser::nodes, blender::gpu::shader::parser::LexerBase::reset_template_tokens(), and bsl::BSLParser::translation_unit().
Referenced by blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::parse().
| void blender::gpu::shader::parser::ParserBase::print_ast | ( | ) | const |
Definition at line 1355 of file bsl/grammar.cc.
References blender::gpu::shader::parser::ast::Node::print_ast().
| ast::LocalScope blender::gpu::shader::parser::ParserBase::root | ( | ) | const |
Definition at line 1360 of file bsl/grammar.cc.
References blender::gpu::shader::parser::ast::LocalScope.
Referenced by blender::gpu::tests::TestParser::root().
| std::vector<ast::NodeData> blender::gpu::shader::parser::ParserBase::ast_nodes |
Flat array of nodes (nodes pointers are indices to this array).
Nodes are stored in source order (not in logical execution order). This allow to have range based iterators follow the source order. Expressions need to use a parser to use logical execution order with operator precedence.
Definition at line 110 of file token_stream.hh.
Referenced by blender::gpu::shader::parser::ast::Node::back(), blender::gpu::shader::parser::ast::Node::child_first(), blender::gpu::shader::parser::ast::Node::child_last(), blender::gpu::shader::parser::ast::Node::data(), blender::gpu::shader::parser::ast::Node::front(), blender::gpu::shader::parser::ast::Node::has_single_child(), blender::gpu::shader::parser::ast::Node::is_empty(), blender::gpu::shader::parser::ast::Node::next(), blender::gpu::shader::parser::ast::Node::parent(), parse_bsl(), blender::gpu::shader::parser::ast::Node::prev(), and blender::gpu::shader::parser::ast::Node::type().
| int blender::gpu::shader::parser::ParserBase::include_id = 0 |
Definition at line 85 of file token_stream.hh.
| std::vector<ScopeLinks> blender::gpu::shader::parser::ParserBase::scope_links |
Index of adjacent scopes.
Definition at line 97 of file token_stream.hh.
Referenced by build_scope_tree().
| std::vector<IndexRange> blender::gpu::shader::parser::ParserBase::scope_ranges |
Range of token per scope.
Definition at line 95 of file token_stream.hh.
Referenced by build_scope_tree(), and build_token_to_scope_map().
| std::vector<ScopeType> blender::gpu::shader::parser::ParserBase::scope_types |
Range of token per scope.
Definition at line 93 of file token_stream.hh.
Referenced by build_scope_tree(), and blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::operator()().
| std::string_view blender::gpu::shader::parser::ParserBase::scope_types_str |
Compact visualization of scope_types.
Definition at line 88 of file token_stream.hh.
Referenced by blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::debug_print(), and blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::debug_print_tokens().
| std::vector<int> blender::gpu::shader::parser::ParserBase::token_scope |
Index of bottom most scope per token.
Definition at line 99 of file token_stream.hh.
Referenced by build_token_to_scope_map(), blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::debug_print(), and blender::gpu::shader::parser::IntermediateForm< parser::FullLexer, parser::FullParser >::debug_print_tokens().