lexer change

[Abdur-rahmaanJ]

why not use a hard, non-regex lexer?

[sidd607]

Because regex based lexer seemed simpler to implement at that time. How do you use a non-regex lexer?