Page 1 of 1

ReadLine() »

Posted: Wed Oct 27, 2021 7:08 pm
by Flinx
Hello,
I noticed that text lines starting with the » character cause the line not to be read in correctly.
The ReadLine function generally omits the second byte if it is a 0xBB if the line has only two bytes. With more bytes strange things happen.
I suspect this has to do with searching for the BOM, although it not only occurs with the first line.

Code: Select all

Function ReadLineTest()
	OpenFile(1, "ReadLineTest.txt")
	rdline$=ReadLine(1)
	DebugPrint(rdline$)
	hx$=""
	For Local i=0 To StrLen(rdline$, #ENCODING_RAW)-1
		hx$=hx$ .. MidStr((HexStr(ByteAsc(rdline$, i))),1).." "
	Next
	DebugPrint(hx$)
EndFunction

StringToFile("  ABC 1 » ä-ö-ü-ß-Ä-Ö-Ü","ReadLineTest.txt")
ReadLineTest()
StringToFile("»ABC 2 » ä-ö-ü-ß-Ä-Ö-Ü","ReadLineTest.txt")
ReadLineTest()
StringToFile("»","ReadLineTest.txt")
ReadLineTest()
/*
For k=0x0000 To 0xffff
	StringToFile(ByteStrStr(k,#SHORT), "ReadLineTest.txt")
	ReadLineTest()
Next
*/
Ralf

Re: ReadLine() »

Posted: Fri Oct 29, 2021 9:55 pm
by airsoftsoftwair
Yup, clearly a bug but fixed now. As a workaround, just open the file in ISO 8859-1 encoding by passing the "Encoding" tag to OpenFile(). Then the problem shouldn't occur.

Code: Select all

- Fix: When opening a file in UTF-8 mode, ReadLine() could get confused when reading a byte that is part
  of the UTF-8 BOM ($ef, $bb, $bf)