Better UTF-8 and tabs support #31
Conversation
it's easier to debug this way. This commit can be easily reverted
|
I didn't recognize it earlier but these changes breaks cellular automaton :) Because now grid elements can have different display width there are many side effects. E.g. look at this picture (tabs were replacing with corresponding amount of "orange" stars). You might clearly see than even if line (let's call it line A) is visually longer that the line below symbols are not falling down because their columns indices in grid are less (because line below contains less multicell symbols even if it looks shorter it's actually has more cell filled in the grid).
Slide animation (the one from README) if also looks not as expected (when multicell symbols disappears on the right side they're not instantly showing yourselves on left left side) While I'm still liking how emojis and other non-ASCII symbols are falling down (even with the problem i described above), there's definitely a room for upgrades. |
|
Now I decided (to not break cellular automaton logic) to replace all multicolumn symbols (and also tabs) with corresponding amount or replacers (now it's a @ symbol with WarningMsg highlight group, spaces are replacers for tab symbol). E.g. buffer this this content
will be be replaced to this one:
Then it would fall gracefully as it fell earlier since all bytes are 1-column width ones. P.S. It's not that cool as it could be :) But dealing with cellular automaton where different cells can have different size (width in this case) is a hell of a math problem and it definitely doesn't fit to this plugin's concept. P.P.S. Since all bytes are 1-column width ones all changes made in ui.lua can also be reverted |
Now I'm asking vim (see *:h virtcol()*) "tell me what virtual columns the symbol at [lineno, col] occupies" and the I can gracefully append it to grid (e.g. check if multicolumn symbol like tab or emoji can be fully displayed or maybe it's leading part is to the left of the leftmost column; also replace it with corresponding amount of replacers) Buffers with leftcol > 0 are also handling properly now.
|
P.P.P.S. Changed the way how I'm iterating over symbols in line (virtcol() and strcharpart() were very useful), also fixed grid filling where winsaveview().leftcol > 0. Even though I had time to make these changes I feel like I will not be able to write tests until weekend (still now nothing about Lua test frameworks but I saw your tests and your setup_viewport function, shouldn't be a big deal to write some). Any of your feedback will be helpful to me. Maybe you have another point of view on this |
actually it was made in some of previous commits but for somehow I decided to revert it ...
|
Hey, thanks for your contribution. And sorry for slow response, but I'm going through some busy time in my life. I'll try to look into your PR later this week. Don't worry about the tests for now. They have been broken since neovim update and I wasn't able to figure out how to fix them. |
That was a reason why tests fail (virtcol always returned [0, 0] and grid wasn't filled)
|
Looks like you didn't have time last month and I also forgot about this PR as well :) I fixed all existing test fails. The strangest one was "stack overflow" on vim.treesitter table reverting, for now I made stub only for get_captures_at_pos() function. |
|
Looks like everything works, at least as I expected, Now it's up to you to check things up. I'm not rushing you in any way, for now I just can live in my fork :) |
Closes #30.
Better UTF-8 support because now we're iterating over UTF-8 symbols (which can be up to 4 bytes long) when initializing grid. Also there's strdisplaywidth() calls under the hood to properly calculate how many cells will be occupied by each of grid cells to prevent nvim lines to wrap.
strdisplaywidth() also respects tabstop which is nice - i checked it with make_it_rain animation and it looks fine IMO, but slide animation (the one from README example) looks a bit silly when buffer contains tabulations :) The other options is to pre-calculate every tab width on initial lines respecting tabstop and replace it with corresponding amount of spaces to make animations "less dynamic" and more predictable.
I also made a check if buffer fileencoding is UTF-8 but maybe it is not that necessary. Previously this plugin didn't handle multibyte-chars good enough - so if buffer is not UTF8-encoded then the animation will be probably not good as it is for UTF-8 buffers before the changes I'm proposing.
Another thing I made was 'list' option enabling (only tab option was set in 'listchars') - it was easier for me to debug how tabs are behaving on each animation frame - also it's easy to revert.
This PR is raw, no tests was written (never used luassert/busted before), probably i will have a free time this weekend to write some