bug: Incorrect handling/rendering of emoji ( CJK/ZWJ, BMP, etc )

Hey! Let me read a little bit into this to see what I think. I would think rendering emojis should more or less just work as is. Do you have a simple example of it not working?

EDIT: Oh I see the issue. Let me dig into this and see what I can come up with

Hey heyy! Sorry, lotta late nights and what not. Ummm, I don't know if I still have example that isn't unmodified or butchered in some way still. I started out with exactly that... cause figured you would wanna see but I ended up kinda going over the deep end and it broke me, drove me mad. It wouldn't be too hard to knock up again though..

Basically I have a bottom full width box framed input panel that expands upwards for multiline input and bracketed paste, with a right hand box framed sidebar on top of it and then the remaining left viewport area was scrollable panel for terminal event flow where messages would appear in box frames. Think uh, basic chat app or such. I'll see about remaking demo again when I recover a bit.

Then I just used generated event messages with all variety different types and amounts of symbols and emojis.. flags, CJK, ZWJ, BMP etc. While also mixing in same messages manually through input panel.

Like I said, I seen where they were defined and what not and tried adding symbols and ranges that were breaking layout or warping word wrap and throwing off cursor, tried adjusting widths, but idk.. I just never could get it right. Fk unicode.

Thanks a lot for looking into it, feel good knowing you're on the case! o7

Awesomesauce! Wow, that was fast.. Looks similar to what I ended up doing just minus the extra package. Looking forward to trying it out soon, thank you <3.

I actually have two competing solutions I'm playing with that have different architectural implications. I want to make sure I test with a setup that broke before. Do you have a minimal example of what you were seeing that made you want to fix it?

Here's what I have currently, I don't have clean version before I switched over to the runeseg unfortunately but it really shouldn't take much to swap things around again, it's just single file demo of layout with minimal features and a /test-scroll command that generates simulated events incoming. Just using replace directive in go.mod so should be pretty easy to fix up.

The only ones still breaking on my end with runeseg are the skin tones..

https://github.com/ST33LDI9ITAL/go-tui_demo

I can do a pull on what you got and swap and fix after lil while.

Ok I updated it to use new grapheme branch.
Can we get stringWidth exported?
Couple of the things needed or nice to have are unexported (nextCluster, stringWidth, wrapText)
Currently kinda broke it due to that.

Additionally, but better fit to new issue for unexported would be:

reflect + unsafe are used for 3 things in tui-test:

1. Inserting newlines at cursor (lines 55-59) — insertNL() reads textArea.cursorPos (a *State[int]) to insert \n at the
   correct grapheme position. No public API for this — TextArea has no InsertAt(pos, string) or cursor getter.
2. Adjusting match panel height (lines 374-375) — sets matches.style.Height = Fixed(mh + 2) via unsafe pointer. Element
   options are only applied at creation time — no SetHeight() method exists after creation.
3. Cursor positioning in PostRenderHook (lines 408, 445) — reads textArea.cursorPos and textArea.tempScrollOffset to
   compute the terminal cursor position for the external cursor (not go-tui's built-in cursor). TempScrollOffset accounts
   for the textarea's internal scroll when input extends beyond its visible area.

All three would be eliminated by exporting:

• TextArea.CursorPos() int or a public cursor getter
• Element.SetHeight(Value) method
• TextArea.ScrollOffset() int for the internal temp scroll

I asked ye ol Clanktavius to reflect on everything done on project and tests so far and make a list of exported/api nice to haves for go-tui, can post that up after get all this sorted out. Quite a few are pretty legit.

A lot of the hackiness comes from the multiline input and cursor I did.

Hey Grindlemire,

Alright, took a stab at it. Added onto what you provided, compared with what I had, and runeseg, and overall seems pretty complete and correct as far as I can tell. Passed tests, passing on the demo app, and using in my main app now without issue but with limited testing so far and all seems ok.

Would def like you to check and review at some point please.

Also, if you consider PR then please let me know if and how you would like it squashed.

• Edit -
  I spoke too soon, there's still artifact trying to trace out..

o7

I pushed my version up as #97. It adds a small grapheme segmenter and runs measurement, wrapping, rendering, and the editable widgets through it, so a cluster gets measured and drawn as one glyph. I also pulled in the VS15 fix, the ANSI-aware wrap, and the edge tests from your #103.

Would you mind pulling it and trying it against the stuff that was breaking for you? That's what I want to confirm before merging. If anything still looks off, drop the string here and I'll dig in.

Hmm, ok well yours still has artifacts and right border offset pushing with /test-scroll but status bar is ok. Mine renders /test-scroll cleanly but has a status bar artifact at start and on input new lines.

Gonna compare and tinker with both lil bit.. got a interactive pty/term setup now that agent can use for debugging and testing to play with. Hopefully help sort this out.

Also, this has me thinking.. should go-tui maybe detect what terminal is being used and have modes ( modern / compat ) or feature flags: grapheme aware/width checking, vs15/16, ANSI only and modern terminal, colors, etc?

Something like:
Mode: Modern/ANSI/Compat/Auto
Width: Cell/Grapheme/Auto
Color: ANSI/256/True/Auto
Unicode: Standard/Modern/VS15/VS16/Auto

Cause like ghostty and modern terminals handle differently than others and then you have ssh, tmux, screen, etc as well. So think it might would be beneficial considering apps made with it may be running in different environments, over ssh, or on minimal distro or something.

Finally. Seems good now. #103