Scintilla: use UTF-8 decoding to convert font name to Unicode

[josephsl]

Link to issue number:

Fixes #9892

Summary of the issue:

Format info command fails in Notepad++ and other Scintilla controls because font name is a bytes-like object.

Description of how this pull request fixes the issue:

Font name is decoded using UTF-8.

Testing performed:

Tested with Python 3 source code version of NVDA and notepad++, ensuring font name and other attributes are announced.

Known issues with pull request:

None

Change log entry:

None

Additional context:

Originally, text utils was considered, but passing buffer.raw results in NULL characters being included in format info output. Thus directly decode buffer.value in UTF-8 mode.

[LeonarddeR]

I think using textUtils is a bit overkill here, I think you can just decode the value from the string buffer from utf_8. this function is textUtils is mainly there to support cases where we have to deal with raw buffers, containing null characters. The scintilla docs say that the font name string is null terminated, though.

In fact, what you're doing now is probably going to include null characters in the output for the font name. For this not to happen, you will need to know the actual number of characters for the font in the buffer.

If you insist on using textUtils, see my comment below, but I think it is unnecessary.

[LeonarddeR]

From the docs:

Under Windows, only the first 32 characters of the name are used, the name is decoded as UTF-8, ...

Suggested change

	formatField["font-name"]=textUtils.getTextFromRawBytes(fontNameBuf.raw,numChars=fontNameLength)
	formatField["font-name"]=textUtils.getTextFromRawBytes(fontNameBuf.raw,numChars=fontNameLength, encoding="utf_8")

[josephsl]

Hi, a simpler solution might be to just do buf.value.encode(“utf-8”) provided that NULL character isn’t included (I’ll run it through I/O log). Thanks for pointing this out.

[LeonarddeR]

exactly that. buf.value will already strip the null char(s) at the end.

…

Op 8 jul. 2019 om 06:23 heeft Joseph Lee ***@***.***> het volgende geschreven: Hi, a simpler solution might be to just do buf.value.encode(“utf-8”) provided that NULL character isn’t included (I’ll run it through I/O log). Thanks for pointing this out. From: Leonard de Ruijter ***@***.***> Sent: Sunday, July 7, 2019 9:14 PM To: nvaccess/nvda ***@***.***> Cc: Joseph Lee ***@***.***>; Author ***@***.***> Subject: Re: [nvaccess/nvda] Scintilla: use text utils module to convert font name to Unicode (#9893) @LeonarddeR requested changes on this pull request. I think using textUtils is a bit overkill here, I think you can just decode the value from the string buffer from utf_8. this function is textUtils is mainly there to support cases where we have to deal with raw buffers, containing null characters. The scintilla docs say that the font name string is null terminated, though. In fact, what you're doing now is probably going to include null characters in the output for the font name. For this not to happen, you will need to know the actual number of characters for the font in the buffer. If you insist on using textUtils, see my comment below, but I think it is unnecessary. _____ In source/NVDAObjects/window/scintilla.py <#9893 (comment)> : > internalBuf=winKernel.virtualAllocEx(self.obj.processHandle,None,len(fontNameBuf),winKernel.MEM_COMMIT,winKernel.PAGE_READWRITE) try: watchdog.cancellableSendMessage(self.obj.windowHandle,SCI_STYLEGETFONT,style, internalBuf) winKernel.readProcessMemory(self.obj.processHandle,internalBuf,fontNameBuf,len(fontNameBuf),None) finally: winKernel.virtualFreeEx(self.obj.processHandle,internalBuf,0,winKernel.MEM_RELEASE) - formatField["font-name"]=fontNameBuf.value + formatField["font-name"]=textUtils.getTextFromRawBytes(fontNameBuf.raw,numChars=fontNameLength) From the docs: Under Windows, only the first 32 characters of the name are used, the name is decoded as UTF-8, ... ⬇️ Suggested change - formatField["font-name"]=textUtils.getTextFromRawBytes(fontNameBuf.raw,numChars=fontNameLength) + formatField["font-name"]=textUtils.getTextFromRawBytes(fontNameBuf.raw,numChars=fontNameLength, encoding="utf_8") — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#9893?email_source=notifications&email_token=AB4AXEBRN46ZCTX7L2YXBADP6K5JZA5CNFSM4H6WVNEKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB5VSKIY#pullrequestreview-258680099> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4AXEARPHZVALMXYGE7SVLP6K5JZANCNFSM4H6WVNEA> . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[josephsl]

Hi,

In the end, I decided to make this simpler through buffer.value.decode call. One way for text utils solution is passing buffer.value, but it is overkill.

Thanks.

[feerrenrut]

Thanks @josephsl