Object.ToString() returns hex E28892 instead of D2 for minus signs on Linux

[SWarnberg]

Issue Title

Invalid unicode character is returned by Object.ToString() when running on Linux for minus.

General

When using Object.ToString() on an object of type Int16, Int32 etc, with a negative value, for instance -32, the return value is "-32". However, this method behaves differently on Windows and Linux, which causes problems when forwarding this string to another environment, for instance SQL Server.

• On Windows, the minus sign is of hex code 2D (Hyphen minus), see https://www.fileformat.info/info/unicode/char/2d/index.htm.
• On Linux, the minus sign is of hex code E28892, see https://www.fileformat.info/info/unicode/char/2212/index.htm.

This causes problems, for instance when passing a SQL statement to another machine (on Windows) in SQL Server. E28892 is not considered as a negative number in SQL Server. When following is executed:
EXEC sp_myproc @mydata = -32
then the following error message is returned:

Incorrect syntax near '−'.

[leecow]

@danmosemsft - who could help?

[SWarnberg]

I investigated a bit further since this only happens in very rare circumstances. The short background is that we have a web server, with web ui and api's.

When running the web ui where the JavaScript methods executes the web api, it works without a problem. However, when we ran the same web api from Postman, it failed.

Recently I added support for RequestCulture in the web project, to have dates and numbers formatted according to the user's browser locale. The default culture was set to "sv-SE". Since Postman by default is not setting Accept-Language in header, "sv-SE" was used. Apparently "sv-SE" gives above problem, whereas "en-SE" does not. By adding Accept-Language header to "en-SE" in Postman, the request works.

This is very strange to me, since both "sv-SE" and "en-SE" should have the same formatting rules. Also, shouldn't ToString() give the exact result, irrespective of locale, for minus signs?

[SWarnberg]

EncodingAPI.zip
Here's a sample project that shows the problem. Debug the solution in Docker (Linux) and execute http://localhost:10000/api/HexValue in Postman with -2 or something in the Body (raw - application/json).

There are a couple of issues already about different minus signs on Linux vs Windows, here's one closed issue: https://github.com/dotnet/corefx/issues/30993

[danmoseley]

Wierd. @tarekgh any theory?

[stephentoub]

Also, shouldn't ToString() give the exact result, irrespective of locale, for minus signs?

No, it uses https://docs.microsoft.com/en-us/dotnet/api/system.globalization.numberformatinfo.negativesign, which is defined by the culture. If you want to use invariant formatting, you can pass CultureInfo.InvariantCulture to number.ToString.

[tarekgh]

We are using the underlying OS to get the cultures data. It is expected can find some differences between the OS's data. you shouldn't make hard dependency on the culture data as the data can change at any time. As @stephentoub mentioned, if you want to guarantee consistent formatting and parsing, you'll need to use Invariant culture for that. or you can create your own NumberFormatInfo and override the negative sign and pass this object to ToString().

https://docs.microsoft.com/en-us/dotnet/api/system.globalization.numberformatinfo.negativesign?view=netframework-4.8
https://docs.microsoft.com/en-us/dotnet/api/system.int32.tostring?view=netframework-4.8#System_Int32_ToString_System_IFormatProvider_

[tarekgh]

To add more info for this specific case:

CLDR data is using 0x2212 for the negative sign. Windows is converting this sign to regular hyphen. I am following up with Windows team why they are doing that.
https://st.unicode.org/cldr-apps/v#/sv/Symbols/2f08b5ebf85e1e8b

https://unicode.org/cldr/trac/ticket/9851
https://unicode.org/cldr/trac/ticket/10779

[SWarnberg]

Well, as my first comment in this thread; if Windows would change it's behaviour and return 2212, they would need to support it in SQL Server as well.
Also, why does "en-SE" and "sv-SE" give different result? Both are Swedish keyboard layouts.

[SWarnberg]

We are using the underlying OS to get the cultures data. It is expected can find some differences between the OS's data. you shouldn't make hard dependency on the culture data as the data can change at any time. As @stephentoub mentioned, if you want to guarantee consistent formatting and parsing, you'll need to use Invariant culture for that. or you can create your own NumberFormatInfo and override the negative sign and pass this object to ToString().
https://docs.microsoft.com/en-us/dotnet/api/system.globalization.numberformatinfo.negativesign?view=netframework-4.8
https://docs.microsoft.com/en-us/dotnet/api/system.int32.tostring?view=netframework-4.8#System_Int32_ToString_System_IFormatProvider_

Problem is that we're using Object.ToString(), which doesn't have formatting options, since we don't know the datatype. We can of course try to convert it.

[stephentoub]

Problem is that we're using Object.ToString(), which doesn't have formatting options, since we don't know the datatype. We can of course try to convert it.

You can use Convert.ToString(yourObject, CultureInfo.InvariantCulture), which just does the relevant checks for you.