onDataString can mangle multi-byte characters

[nwolverson]

onDataString is easy to use but is unfortunately a subtle foot-gun.

This function calls the underlying node API returning a buffer, and then calls Buffer.toString on the result.

But if a character would span two data events, each Buffer.toString will replace the initial/trailling code units with the replacement character.

As it stands, onDataString should be documented with a clear warning, it is not suitable for general purpose use but only streams that are either guaranteed to have single-byte encoded characters/known short lengths.

The comment on setEncoding unfortunately recommends this:

"Where possible, you should try to use onDataString instead of this function."

I think if this recommended onData, it might be more clear to the user that something fishy is going on.

[jamesdbrock]

If the stream has an end, you can read all the stream data and then convert to a String with https://pursuit.purescript.org/packages/purescript-node-streams-aff/1.1.0/docs/Node.Stream.Aff#v:readAll

If you know how many bytes long the string will be before you read it, then you can wait until all the bytes come in with https://pursuit.purescript.org/packages/purescript-node-streams-aff/1.1.0/docs/Node.Stream.Aff#v:readN