Skip to content

jquast/wcwidth

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Downloads codecov.io Code Coverage MIT License

Introduction

This library is mainly for CLI/TUI programs that carefully produce output for Terminals.

Installation

The stable version of this package is maintained on pypi, install or upgrade, using pip:

pip install --upgrade wcwidth

Problem

All Python string-formatting functions, textwrap.wrap(), str.ljust(), str.rjust(), and str.center() incorrectly measure the displayed width of a string as equal to the number of their codepoints.

Some examples of incorrect results:

>>> # result consumes 16 total cells, 11 expected,
>>> 'コンニチハ'.rjust(11, 'X')
'XXXXXXコンニチハ'

>>> # result consumes 5 total cells, 6 expected,
>>> 'café'.center(6, 'X')
'caféX'

Solution

The lowest-level functions in this library are the POSIX.1-2001 and POSIX.1-2008 wcwidth(3) and wcswidth(3), which this library precisely copies by interface as wcwidth() and wcswidth(). These functions return -1 when C0 and C1 control codes are present.

The iterator functions iter_graphemes() and iter_sequences() allow for careful navigation of grapheme and terminal control sequence boundaries.

An easy-to-use width() function is provided as a wrapper of wcswidth() that is also capable of measuring most terminal control codes and sequences, like colors, bold, tabstops, and horizontal cursor movement.

Finally, text-justification is solved by the grapheme and sequence-aware functions ljust(), rjust(), center(), and wrap(), serving as drop-in replacements to python standard functions of the same names.

Discrepancies

You may find that support varies for complex unicode sequences or codepoints.

A companion utility, jquast/ucs-detect was authored to gather and publish the results of Wide character support and version level, language support, zero-width joiner, and variation-16 support as a General Tabulated Summary by terminal emulator software and version.

Overview

wcwidth()

Use function wcwidth() to determine the length of a single unicode codepoint.

A brief overview, through examples, for all of the public API functions.

Full API Documentation at https://wcwidth.readthedocs.io/en/latest/api.html

wcwidth()

Measures width of a single codepoint,

>>> # '♀' narrow emoji
>>> wcwidth.wcwidth('\u2640')
1

Use function wcwidth() to determine the length of a single unicode character.

See Specification of character measurements. Note that -1 is returned for control codes.

wcswidth()

Measures width of a string, returns -1 for control codes.

>>> # '♀️' emoji w/vs-16
>>> wcwidth.wcswidth('♀️')
2

Use function wcswidth() to determine the length of many, a string of unicode characters.

See Specification of character measurements. Note that -1 is returned if control codes occurs anywhere in the string.

width()

Measures width of a string, with improved handling of control_codes

>>> # same support as wcswidth(), eg. regional indicator flag:
>>> wcwidth.width('\U0001F1FF\U0001F1FC')
2
>>> # but also supports SGR colored text, 'WARN', followed by SGR reset
>>> wcwidth.width('\x1b[38;2;255;150;100mWARN\x1b[0m')
4
>>> # tabs,
>>> wcwidth.width('\t', tabsize=4)
4
>>> # or, tab and all other control characters can be ignored
>>> wcwidth.width('\t', control_codes='ignore')
0
>>> # "vertical" control characters are ignored
>>> wcwidth.width('\n')
0
>>> # as well as sequences with "indeterminate" effects like Home + Clear
>>> wcwidth.width('\x1b[H\x1b[2J')
0
>>> # or, raise ValueError for "indeterminate" effects using control_codes='strict'
>>> wcwidth.width('\n', control_codes='strict')
Traceback (most recent call last):
...
ValueError: Vertical movement character 0xa at position 0

iter_sequences()

Iterates through text, segmented by terminal sequence,

>>> list(wcwidth.iter_sequences('hello'))
[('hello', False)]
>>> list(wcwidth.iter_sequences('\x1b[31mred\x1b[0m'))
[('\x1b[31m', True), ('red', False), ('\x1b[0m', True)]

Use iter_sequences() to split text into segments of plain text and escape sequences. Each tuple contains the segment string and a boolean indicating whether it is an escape sequence (True) or text (False).

iter_graphemes()

Use iter_graphemes() to iterate over grapheme clusters of a string.

>>> from wcwidth import iter_graphemes
>>> # ok + Regional Indicator 'Z', 'W' (Zimbabwe)
>>> list(wcwidth.iter_graphemes('ok\U0001F1FF\U0001F1FC'))
['o', 'k', '🇿🇼']

>>> # cafe + combining acute accent
>>> list(wcwidth.iter_graphemes('cafe\u0301'))
['c', 'a', 'f', 'é']

>>> # ok + Emoji Man + ZWJ + Woman + ZWJ + Girl
>>> list(wcwidth.iter_graphemes('ok\U0001F468\u200D\U0001F469\u200D\U0001F467'))
['o', 'k', '👨\u200d👩\u200d👧']

A grapheme cluster is what a user perceives as a single character, even if it is composed of multiple Unicode codepoints. This function implements Unicode Standard `Annex #29`_ grapheme cluster boundary rules.

ljust()

Use ljust() as replacement of str.ljust():

>>> 'コンニチハ'.ljust(11, '*')             # don't do this
'コンニチハ******'
>>> wcwidth.ljust('コンニチハ', 11, '*')    # do this!
'コンニチハ*'

rjust()

Use rjust() as replacement of str.rjust():

>>> 'コンニチハ'.rjust(11, '*')             # don't do this
'******コンニチハ'
>>> wcwidth.rjust('コンニチハ', 11, '*')    # do this!
'*コンニチハ'

center()

Use center() as replacement of str.center():

>>> 'cafe\u0301'.center(6, '*')             # don't do this
'café*'
>>> wcwidth.center('cafe\u0301', 6, '*')
'*café*'                                    # do this!

wrap()

Use function wrap() to wrap text containing terminal sequences, Unicode grapheme clusters, and wide characters to a given display width.

>>> from wcwidth import wrap
>>> # Basic wrapping
>>> wrap('hello world', 5)
['hello', 'world']

>>> # Wrapping CJK text (each character is 2 cells wide)
>>> wrap('コンニチハ', 4)
['コン', 'ニチ', 'ハ']

>>> # Text with ANSI color sequences
>>> wrap('\x1b[31mhello world\x1b[0m', 5)
['\x1b[31mhello', 'world\x1b[0m']

The SequenceTextWrapper class extends :class:`textwrap.TextWrapper` for sequence-aware wrapping with full control over wrapping behavior.

Full API Documentation at https://wcwidth.readthedocs.io

Developing

Install wcwidth in editable mode:

pip install -e .

Execute all code generation, autoformatters, linters and unit tests using tox:

tox

Or execute individual tasks, see tox -lv for all available targets:

tox -e pylint,py36,py314

Updating Unicode Version

Regenerate python code tables from latest Unicode Specification data files:

tox -e update

The script is located at bin/update-tables.py, requires Python 3.9 or later. It is recommended but not necessary to run this script with the newest Python, because the newest Python has the latest unicodedata for generating comments.

Building Documentation

This project is using sphinx 4.5 to build documentation:

tox -e sphinx

The output will be in docs/_build/html/.

Updating Requirements

This project is using pip-tools to manage requirements.

To upgrade requirements for updating unicode version, run:

tox -e update_requirements_update

To upgrade requirements for testing, run:

tox -e update_requirements38,update_requirements39

To upgrade requirements for building documentation, run:

tox -e update_requirements_docs

Utilities

Supplementary tools for browsing and testing terminals for wide unicode characters are found in the bin/ of this project's source code. Just ensure to first pip install -r requirements-develop.txt from this projects main folder. For example, an interactive browser for testing:

python ./bin/wcwidth-browser.py

Uses

This library is used in:

Other Languages

There are similar implementations of the wcwidth() and wcswidth() functions in other languages.

History

0.3.0 2026-01-18
0.2.14 2025-09-22
  • Drop Support for Python 2.7 and 3.5. PR #117.
  • Update tables to include Unicode Specifications 16.0.0 and 17.0.0. PR #146.
  • Bugfix U+00AD SOFT HYPHEN should measure as 1, versions 0.2.9 through 0.2.13 measured as 0. PR #149.
0.2.13 2024-01-06
  • Bugfix zero-width support for Hangul Jamo (Korean)
0.2.12 2023-11-21
  • Bugfix Re-release to remove .pyi files misplaced in wheel Issue #101.
0.2.11 2023-11-20
  • Updated Include tests files in the source distribution (PR #98, PR #100).
0.2.10 2023-11-13
  • Bugfix accounting of some kinds of emoji sequences using U+FE0F Variation Selector 16 (PR #97).
  • Updated Specification.
0.2.9 2023-10-30
  • Bugfix zero-width characters used in Emoji ZWJ sequences, Balinese, Jamo, Devanagari, Tamil, Kannada and others (PR #91).
  • Updated to include Specification of character measurements.
0.2.8 2023-09-30
  • Include requirements files in the source distribution (PR #82).
0.2.7 2023-09-28
  • Updated tables to include Unicode Specification 15.1.0.
  • Include bin, docs, and tox.ini in the source distribution
0.2.6 2023-01-14
  • Updated tables to include Unicode Specification 14.0.0 and 15.0.0.
  • Changed developer tools to use pip-compile, and to use jinja2 templates for code generation in bin/update-tables.py to prepare for possible compiler optimization release.
0.2.1 .. 0.2.5 2020-06-23
  • Repository changes to update tests and packaging issues, and begin tagging repository with matching release versions.
0.2.0 2020-06-01
  • Enhancement: Unicode version may be selected by exporting the Environment variable UNICODE_VERSION, such as 13.0, or 6.3.0. See the jquast/ucs-detect CLI utility for automatic detection.
  • Enhancement: API Documentation is published to readthedocs.io.
  • Updated tables for all Unicode Specifications with files published in a programmatically consumable format, versions 4.1.0 through 13.0
0.1.9 2020-03-22
  • Performance optimization by Avram Lubkin, PR #35.
  • Updated tables to Unicode Specification 13.0.0.
0.1.8 2020-01-01
  • Updated tables to Unicode Specification 12.0.0. (PR #30).
0.1.7 2016-07-01
  • Updated tables to Unicode Specification 9.0.0. (PR #18).
0.1.6 2016-01-08 Production/Stable
  • LICENSE file now included with distribution.
0.1.5 2015-09-13 Alpha
  • Bugfix: Resolution of "combining character width" issue, most especially those that previously returned -1 now often (correctly) return 0. resolved by Philip Craig via PR #11.
  • Deprecated: The module path wcwidth.table_comb is no longer available, it has been superseded by module path wcwidth.table_zero.
0.1.4 2014-11-20 Pre-Alpha
0.1.3 2014-10-29 Pre-Alpha
0.1.2 2014-10-28 Pre-Alpha
0.1.1 2014-05-14 Pre-Alpha
  • Initial release to pypi, Based on Unicode Specification 6.3.0

This code was originally derived directly from C code of the same name, whose latest version is available at https://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c:

* Markus Kuhn -- 2007-05-26 (Unicode 5.0)
*
* Permission to use, copy, modify, and distribute this software
* for any purpose and without fee is hereby granted. The author
* disclaims all warranties with regard to this software.

About

Python library that measures the width of strings in a terminal

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published