Page 1 of 1

Using generated LALRGen parser as Scilexer

Posted: 9 May 2021 10:02
by sbishop61
coming back to VIP after many years Vip6CE.
I was wondering whether/how you can use a generated parser as a Scilexer for the edit control?
I would like to use a generated parser:
  • Parse text in editor window with error reporting
    Colour/format text in window
    Both on opening a file and on a change

Re: Using generated LALRGen parser as Scilexer

Posted: 10 May 2021 11:34
by Thomas Linder Puls
A genuine (C++) lexer for sciLexer is capable of token coloring fragments of text. Briefly it works in the following way. The lexer is a state machine each character in the editor transfers from one state to another. The state that a character leads to is saved with the character. Each state is also given a style (color/font,etc). When modify the editor you can pick-up from the preceding character (here is also a line state so you will start from the start of the line).

The lexers that vipLalrGen produces does not have such capabilities so they are not directly usable.

However, the lexers that vipLalrGen produces have a number of properties that fits extremely well for Visual Prolog. For example numbers, string literals, comments, identifiers, etc match 100% (guess why) to Visual Prolog. This also means that the the Visual Prolog sciLexer fits really well to any language you may define using vipLalrGen.

The main difference in the lexer side of such languages will be which keywords the language has.

If you call lexerDefault_visualProlog and then postAction setting of your own keywords then the lexer side should work:

Code: Select all

    editor:lexerDefault_visualProlog(),     postAction({ :-         editor:setKeywords(0, "go for it"), % major keywords         editor:setKeywords(1, ""), % minor keywords         editor:setKeywords(2, ""), % directive keywords         editor:setKeywords(3, ""), % doc-keywords     }),
Regarding parsing and error reporting. Here you will have to retrieve the entire content of the editor and parse it. So it is more "heavy" operation which should not be performed "too" often. You may try it on an updateUIEvent event:

Code: Select all

properties     updateUIEvent : event1{unsigned UpdatedFlag} (o).     % SCN_UPDATEUI     % Either the text or styling of the document has changed or the selection range or scroll position has changed. Now would     % be a good time to update any container UI elements that depend on document or view state. The updated field is set to     % the bit set of things changed since the previous notification.     %     % Symbol Value Meaning     %     % SC_UPDATE_CONTENT 0x01 Contents, styling or markers have been changed.     % SC_UPDATE_SELECTION 0x02 Selection has been changed.     % SC_UPDATE_V_SCROLL 0x04 Scrolled vertically.     % SC_UPDATE_H_SCROLL 0x08 Scrolled horizontally.     %

Re: Using generated LALRGen parser as Scilexer

Posted: 18 May 2021 9:49
by sbishop61
Thank you