parse failed for mutibytes characters, encode will show in \xxx

[zhouronghuaztecomcn]

BPO	33684
Nosy	@vstinner, @ezio-melotti, @methane, @miss-islington, @zhou.ronghua@zte.com.cn
PRs	bpo-33684:when strings or files are encode with utf8 without bom, auto detect encoding will fail. set default decoded type with utf-8 for most time #7203 bpo-33684:parse failed for mutibytes characters, encode will show in \xxx #7286 bpo-33684: json.tool: Use utf-8 for infile and outfile. #17460 [3.8] bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460) #17464 [3.7] bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460) #17465

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-12-04.10:27:07.562>
created_at = <Date 2018-05-29.14:29:39.270>
labels = ['3.7', '3.8', 'type-bug', 'library', '3.9']
title = 'parse failed for mutibytes characters, encode will show in \\xxx'
updated_at = <Date 2019-12-04.10:27:17.745>
user = 'https://github.com/zhouronghuaztecomcn'

bugs.python.org fields:

activity = <Date 2019-12-04.10:27:17.745>
actor = 'methane'
assignee = 'none'
closed = True
closed_date = <Date 2019-12-04.10:27:07.562>
closer = 'methane'
components = ['Library (Lib)']
creation = <Date 2018-05-29.14:29:39.270>
creator = 'zhou.ronghua'
dependencies = []
files = []
hgrepos = []
issue_num = 33684
keywords = ['patch']
message_count = 4.0
messages = ['318039', '357786', '357789', '357791']
nosy_count = 5.0
nosy_names = ['vstinner', 'ezio.melotti', 'methane', 'miss-islington', 'zhou.ronghua']
pr_nums = ['7203', '7286', '17460', '17464', '17465']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33684'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[zhouronghuaztecomcn]

when type this command in windows(xp or win7, all the same):
python -m json.tool xxx.txt xxx.json
if xxx.txt contains Chinese(or other multibytes characters):
if xxx.txt is encoded in ansi, xxx.json will encode Chinese as \xxx, very bad to see what they are;
if xxx.txt is encoded in utf8(without bom for most of the time), because with no bom, json.tool will think it is encoded in ansi, and decode fail.

as now, utf8 is widely use, set default to utf8 for most of the time when auto detect encoding failed

[methane]

New changeset 808769f by Inada Naoki in branch 'master':
bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460)
808769f

[miss-islington]

New changeset a75cad4 by Miss Islington (bot) in branch '3.8':
bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460)
a75cad4

[methane]

New changeset e0f148e by Inada Naoki in branch '3.7':
bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460)
e0f148e