IDLE font settings: use multiple character sets in examples

[terryjreedy]

BPO	13802
Nosy	@terryjreedy, @serwy, @rovitotv, @serhiy-storchaka, @mlouielu, @corona10
PRs	bpo-13802: IDLE: Add pangram and chinese sentence to font sample box #2616 bpo-13802: Use non-Latin characters in IDLE's Font settings sample. #3960 [3.6] bpo-13802: Use non-Latin characters in IDLE's Font settings sample. (GH-3960) #4027
Files	issue13802.patch: Some more example chars

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2017-10-17.23:52:32.369>
created_at = <Date 2012-01-16.23:22:19.711>
labels = ['expert-IDLE', 'type-feature', '3.7']
title = 'IDLE font settings: use multiple character sets in examples'
updated_at = <Date 2017-10-17.23:52:32.368>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2017-10-17.23:52:32.368>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2017-10-17.23:52:32.369>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2012-01-16.23:22:19.711>
creator = 'terry.reedy'
dependencies = []
files = ['29553']
hgrepos = []
issue_num = 13802
keywords = ['patch']
message_count = 19.0
messages = ['151415', '185080', '297868', '297869', '297870', '297878', '297883', '298408', '298413', '298762', '304205', '304210', '304212', '304219', '304222', '304223', '304293', '304543', '304545']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'roger.serwy', 'Todd.Rovito', 'francismb', 'serhiy.storchaka', 'louielu', 'corona10']
pr_nums = ['2616', '3960', '4027']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue13802'
versions = ['Python 3.6', 'Python 3.7']

[terryjreedy]

In the Fonts/Tabs tab of the IDLE Preference dialog, the large box for examples of the font selected shows a small square of ascii chars. I think the box should also show 1 char for each of several alphabets so the consequence of choosing various fonts will be more evident. I am thinking of adding several line with the format
Alphabet \uXXXX <char>

[francismb]

Hi Terry,
just take/put away some ... (they're not in a special order nor preference, just some that could 'see' in the browser).

[mlouielu]

Add the string with pangram and chinese, now sample text shows:

AaBbCcDdEe
FfGgHhIiJjK
1234567890
#:+=(){}[]

The quick brown fox jumps over the lazy dog. 1

南去經三國，東來過五湖 2

----------------------

[serhiy-storchaka]

There are many different alphabets in the world. Why Chines but not Cyrillic or Devanagari? An example that includes all scripts would be too large (and likely most characters would not be rendered correctly with an arbitrary font).

I suggest just make the box for examples editable and save entered examples in the configuration files.

[mlouielu]

Serhiy: Or would it be to detect the user language environment, and come up with user language example sentence? What I remember that Windows font preview will change the sentence in the different language.

[serhiy-storchaka]

I don't think that it is worth to include in Python distribution examples for a hundred of languages. If there are some system-wide collections of examples we can use them, but this should be platform specific and not always available.

[serhiy-storchaka]

On other hand, we can use the standard font chooser dialog. But it doesn't allow to configure the sample text at all (at least on X Window).

Terry, try please the following commands on Windows:

import tkinter
root = tkinter.Tk()
root.tk.call('tk', 'fontchooser', 'show')

How it looks?

[terryjreedy]

I made this issue a dependency of bpo-24776, which is about redoing the whole font page.

The fontchooser has good, inappropriate for IDLE, and bad points. I consider it an alternate mockup proposal for the font page. For this issue: displaying each font name in the font is cute, but I am not sure I want to imitate. I address the sample selection below.

This issue is about making it evident that IDLE is a BMP unicode editor, not just an ASCII editor; and about showing the consequence of font choices on a particular OS and machine. Expanding the static sample with example of the top N scripts, with N about 10, will do this.

When I looked at Francis's patch, I thought it deficient in that people would not know what chars were being replaced by boxes. Then I tried the patch and none of them were. At least not on Windows. Since I opened this, we added back the Help button. Added help text for this tab can list the scripts represented in the sample. If nothing else, I will use this patch, as it improves of the status quo. I should have done this years ago. An immediate improvement would be all chars from a script on one line and some (more?) hanji/kanji CJK chars.

bpo-24776 suggests putting the sample beside the font selection box. The font box only needs 75% of the width it has. With the frame around the sample label removed, there may be more width available. There will certainly be more lines.

Louie, I understand your PR to be a suggestion about pangrams, expressed in code, to be 'pulled' into my mind, and possibly into my clone, and not a request to merge as is with only Chinese. Others have noted that submitting work-in-progess patches as PRs, rather than as diffs to the tracker, can be confusing. Yet it makes review easy.

In considering the idea, I looked at https://en.wikipedia.org/wiki/Pangram and also found http://clagnut.com/blog/2380/, which lists pangrams in multiple languages that were once on wiki/List_of_Pangrams.

I am rejecting the idea as is for multiple related reasons. The sample is about scripts, not languages. Long phrases mean fewer scripts. Many scripts are used for multiple dialects or even languages. Which one to choose? An innocent or poetic phrase in one language may be less innocent in another dialect/language, or if interpreted metaphorically, or one consider possible alternate meanings of words.

One thing I would consider is script names written in the script. This would replace an arbitrary sample from the same script and could be done on a script-by-script basis. I believe 'devanagari' in Devanagari should be intelligible in all or most north Indian languages that use Devanagari or derivatives thereof, and offensive in none. But I would enquire.

What do others think of this?

Louie, I presume 'hanji' is a single char. How about using the stroke pangram 永 (all basic strokes) in a phrase such as 'hanji character 永'? (I like the look of the char as well).

Serhiy, is there one non-controversial pan-slavic way to write 'Cyrillic' in Cyrillic? Or controversy-engendering national variations?

A separate issue could add a button and dropdown multiple selection list to print to an output window all or a selection of the 256 blocks of 256 codepoints in the BMP. This should eliminate any need for a configuration option.

likely most characters would not be rendered correctly with an arbitrary font

A goal of this issue is to let people see such problems where they exist.

The fontchooser sample awful. Only couple of ascii and script chars and a small sample of script that changes with each script. It has no knowledge of default characters.

[serhiy-storchaka]

Serhiy, is there one non-controversial pan-slavic way to write 'Cyrillic' in Cyrillic?

No. Even in very close east-slavic languages it is written differently: Ukrainian "Кирилиця", Russian "Кириллица", Belarusian "Кірыліца".

[terryjreedy]

I decided to rearrange page bpo-24776 before changing sample bpo-13802.

[terryjreedy]

I decided to do the needed rearrangement of frames in this issue, leaving revision of widgets other than the sample to bpo-24776.

PR3960 initially puts the following in the sample box. I believe it should cover most Python users.

<ASCII/Latin1>
AaBbCcDdEeFfGgHhIiJj
1234567890#:+=(){}[]
¡¢£¥§©«®¶½ÀÁÂÃÄÅÇÐØß

<IPA,Greek,Cyrillic>
ɐɕɘɞɟɤɫɮɰɷɻʁʃʆʎʞʢʫʭʯ
ΑαΒβΓγΔδΕεΖζΗηΘθΙιΚκ
БбДдЖжПпФфЧчЪъЭэѠѤѬӜ

<Hebrew, Arabic>
אבגדהוזחטיךכלםמןנסעף
ابجدهوزحطي٠١٢٣٤٥٦٧٨٩

<Devanagari>
०१२३४५६७८९अआइईउऊएऐओऔ
कगङचजञतदनपबमयरलवशसह॥

<East Asian>
〇一二三四五六七八九
汉字漢字人木火土金水
ᅡᅦᅩᆨᆫ가결걵곴극
あいうえおかさたなま

The new help message explains

Font sample: This shows what a selection of BMP unicode characters look
like for the current font selection. If a font face does not define a
character, Tk attempts to find another font that does. Substitute
glyphs will not necessarily have the same size as the font selected.
Hebrew and Arabic letters should display right to left, starting with
alef, \u05d0 and \u0627. Arabic numerals display left to right. The
East Asian samples are Chinese digits followed by Chinese Hanzi, Korean
Hangul, and Japanese Hiragana.

Except for Chinese, I intend for the samples to be meaningless subsequences of the respective character set. I will try to get comments from others for the Indian, Korean, and Japanese samples.

The Chinese reads 'hanzi hanzi person wood fire earth metal water' with 'hanzi' repeated in simplified and traditional character variations. Louie, could this be controversial in any way?