Opened 9 months ago

Closed 5 months ago

#135 closed defect (fixed)

Cavern tokenisation oddity

Reported by: Olly Betts Owned by: Olly Betts
Priority: minor Milestone: 1.4.11
Component: cavern Version: git master
Keywords: Cc:

Description

I wonder if I should adjust how cavern tokenises. Currently a token is defined as a sequence of consecutive letters, so this is accepted: 

*fix2 reference0 0 0

We don't document this as working, and I doubt anyone is deliberately using it, but it's possible there are a few accidental uses. If we're going to change this we probably ought to make it a warning and give people a chance to fix any real world instances.

Seen in current git master, but I believe it's been like this forever.

Change History (2)

comment:1 Changed 5 months ago by Olly Betts

Milestone: → 1.4.11

comment:2 Changed 5 months ago by Olly Betts

Resolution: → fixed
Status: new → closed

Fixed. [05b9de762d0a3238b35375a3a48ebe3dc87c07d9] adds a warning: 

oddtokenise.svx:1:5: warning: No blank after token
 *fix2 reference0 0 0
     ^
oddtokenise.svx:1:16: warning: No blank after token
 *fix2 reference0 0 0
                ^

I've also pushed changes to read a token including alphanumerics in cases where it can't be followed by a number, to give better error reporting in those cases.

Note: See TracTickets for help on using tickets.