Add end position information to BNFC'Position

[aabounegm]

I made an attempt at implementing the feature I asked for. Please let me know if there is anything I can improve on or some style I'm not strictly following (nitpicks are welcome 😄).

Resolves #461

[aabounegm]

I have tried running the doctests locally but failed for some reason. stack test doesn't seem to run them (I don't have cabal), and installing manually (using stack install doctest) and then running doctest source/src gives me a bunch of errors I couldn't fix, like so:

source\src\BNFC\License.hs:7:1: error:
    Could not find module ‘Data.String.QQ’
    Perhaps you meant Data.String (from base-4.18.1.0)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

source\src\BNFC\CF.hs:38:1: error:
    Could not find module ‘BNFC.Par’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

source\src\BNFC\CF.hs:39:1: error:
    Could not find module ‘BNFC.Lex’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

which also caused a lot of variable not in scope errors. Do you have any suggestions for how to run the doctests?

[aabounegm]

I believe the tests should be fixed now. @andreasabel when do you think you can take a look?

[aabounegm]

Hello @andreasabel, sorry to bother you again, but do you have an estimate for when you might have time to review this?

[andreasabel]

These functions should only be imported when used (currently, that would be under condition functor), otherwise compilation generates warnings with -Wall.

[andreasabel]

What if a token spans several lines? Note that the user can define their own tokens that may contain newline characters.

[andreasabel]

Thanks for the PR, and sorry for the delay in evaluating this PR.
Did you run the testsuite (in subdirectory testing)?
It seems your patch breaks e.g. the --agda --functor backend.
Also, if we change the functionality of --functor, we will potentially break a lot of user code.
So I suggest to implement the new feature under a new flag, e.g. --position, and explain how it differs from --functor.
When I had a quick look at the code I saw that you assume tokens to only span a single line, but user-defined tokens can also span several lines.

[aabounegm]

Thanks for the review, you've raised some good points indeed. Unfortunately, I have gotten rather busy lately and don't think I will be able to address them relatively soon.
About the addition of a new flag, it felt somewhat redundant to have 2 flags with very similar functionality, except one has more information than the other. Wouldn't a major version update be more suitable in your opinion?