Discussion:
Terminal Emulator
(too old to reply)
Gordinator
2024-05-14 17:44:43 UTC
Permalink
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can I
use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.

If you have any advice, please do let me know!
Alan Gauld
2024-05-14 19:06:36 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can I
use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.
The first thing is to decide which terminal. A VT100 is very different
from a 3270. And even a VT330 is quite different from a VT100 although
sharing a common subset of control codes. And if you start looking at
graphical terminals things get even more interesting!

The other thing to consider is whether it will be a standalone app or
a GUI component. If the latter do you want to expose your own API or
clone the manufacturers? Or both?!
Or you could make it an object that can be used both in GUIs and in
"robotic" or "batch" mode. So many options.

Most of the specs are available online and there must be dozens of
terminal emulators around written in C so you should have plenty
of sample code to study. Good luck!
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
Grant Edwards
2024-05-14 19:47:18 UTC
Permalink
Post by Alan Gauld
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly
competent Python user, and I wish to try a new project idea. What
references can I use when writing my terminal emulator? I wish for
it to be a true terminal emulator as well, not just a Tk text
widget or something like that.
The first thing is to decide which terminal.
You also need to decide if you're going to support real serial ports
or just ptys.
Grant Edwards
2024-05-14 20:03:33 UTC
Permalink
Post by Alan Gauld
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly
competent Python user, and I wish to try a new project idea. What
references can I use when writing my terminal emulator? I wish for
it to be a true terminal emulator as well, not just a Tk text
widget or something like that.
The first thing is to decide which terminal.
If you want to make life easier, make it a superset of a terminal that
already exists in the terminfo database.

Going with some sort of ANSI terminal will probably provide
operability even with dumb apps which ignore $TERM and just spit out
basic ANSI escape sequences.

If you really want to break trail, you could invent your own control
sequences, which means you'll have to write terminfo and/or termcap
entries as well as the terminal emulator.
Post by Alan Gauld
A VT100 is very different from a 3270. And even a VT330 is quite
different from a VT100 although sharing a common subset of control
codes. And if you start looking at graphical terminals things get
even more interesting!
"Intersting" is putting it mildly...
Mirko
2024-05-14 20:37:17 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly
competent Python user, and I wish to try a new project idea. What
references can I use when writing my terminal emulator? I wish for
it to be a true terminal emulator as well, not just a Tk text widget
or something like that.
If you have any advice, please do let me know!
true terminal emulator as well, not just a Tk text widget or something like that
If you want to write a GUI terminal, than that *is* a terminal
emulator and *has* a text widget as its visible core. If you want to
write something like getty which runs on the virtual terminals
(Ctrl+Alt+F*) than that is a terminal (not a terminal emulator).

In both cases, you write something that gets input from the
keyboard, processes it and shows the result. How that processing is
done, depends on the terminal standard, like DEC VT{100, 102, 220,
320, etc}.

For a start, you might want to look at Terminator, which is a
terminal emulator written in Python, Gtk and libvte (which does all
the low level stuff).
a***@gmail.com
2024-05-14 21:10:41 UTC
Permalink
The topic was to re-invent the wheel yet again and create a terminal
emulator.

I hesitate to say this but one approach is to consider the curses module as
described by our very own Alan Gauld in a book:

https://www.amazon.com/Programming-curses-Python-Alan-Gauld-ebook/dp/B091B85
B77

The topic is how to make a terminal emulator and as Alan mentions, each kind
of terminal may accept various kinds of escape sequences. There are files
available the are normally used by curses to get a description of sorts of
the capabilities and details of a terminal like a VT100 that curses can use
to decide what stream of bytes to send to update a screen.

You might be able to use something similar, or better, to see what your
terminal emulator should emulate.

And, it may even be possible for you to emulate lots of terminals with the
same basic code.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Alan Gauld via Python-list
Sent: Tuesday, May 14, 2024 3:07 PM
To: Gordinator <***@gordinator.org>; python-***@python.org
Subject: Re: Terminal Emulator
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can I
use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.
The first thing is to decide which terminal. A VT100 is very different
from a 3270. And even a VT330 is quite different from a VT100 although
sharing a common subset of control codes. And if you start looking at
graphical terminals things get even more interesting!

The other thing to consider is whether it will be a standalone app or
a GUI component. If the latter do you want to expose your own API or
clone the manufacturers? Or both?!
Or you could make it an object that can be used both in GUIs and in
"robotic" or "batch" mode. So many options.

Most of the specs are available online and there must be dozens of
terminal emulators around written in C so you should have plenty
of sample code to study. Good luck!
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
--
https://mail.python.org/mailman/listinfo/python-list
Cameron Simpson
2024-05-14 22:35:55 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can
I use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.
Start with the `pty` module.
Stefan Ram
2024-05-15 10:31:25 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python.
We need somethin like a portable curses module (plus colorama)
and it has got to work on both Windoze and Linux straight out
of the box in standard Python. And it would be even sicker if it
could run in that IDLE console too! That way, we could code up
some legit terminal-based software (like slrn or Nethack) using
just plain ol' Python, no extra stuff required. It's a real
shame it ain't already part of the standard library. Someone's
got to step up and make this happen. Python needs a portable
console TUI! It'd be the bomb if you could make /that/ happen!
Gordinator
2024-05-15 11:24:01 UTC
Permalink
Post by Stefan Ram
We need somethin like a portable curses module (plus colorama)
Agreed, getting curses to work on Windows is SUCH a pain, and I don't
think I've ever done it. Naturally, as a Linux user, I don't find much
need to do it anyway.

Colorama would also be cool in the standard library as well. I have
worked on projects in the past where only the standard library could be
used, so I 100% agree with this. But that's something for the PSF to
talk about. Maybe someone could write a PEP for this.
Lawrence D'Oliveiro
2024-05-16 00:12:29 UTC
Permalink
We need somethin like a portable curses module (plus colorama) and
it has got to work on both Windoze and Linux straight out of the box
in standard Python.
Something else for Windows Python users to complain that they cannot get
to install properly?
Gordinator
2024-05-16 18:46:07 UTC
Permalink
Post by Lawrence D'Oliveiro
We need somethin like a portable curses module (plus colorama) and
it has got to work on both Windoze and Linux straight out of the box
in standard Python.
Something else for Windows Python users to complain that they cannot get
to install properly?
To be fair, the problem is the fact that they use Windows (but I guess
Linux users have to deal with venvs, so we're even.
Lawrence D'Oliveiro
2024-05-16 23:07:03 UTC
Permalink
Post by Gordinator
Post by Lawrence D'Oliveiro
We need somethin like a portable curses module (plus colorama) and it
has got to work on both Windoze and Linux straight out of the box in
standard Python.
Something else for Windows Python users to complain that they cannot
get to install properly?
To be fair, the problem is the fact that they use Windows (but I guess
Linux users have to deal with venvs, so we're even.
Linux users get to have standard packages for systemwide installs. You
only need virtual envs for custom setups.

Even nonstandard systemwide installs are nicely contained in /usr/local,
so they are easy to remove.
Peter J. Holzer
2024-05-18 15:19:13 UTC
Permalink
To be fair, the problem is the fact that they use Windows (but I guess Linux
users have to deal with venvs, so we're even.
I don't think Linux users have to deal with venvs any more than Windows
users. Maybe even less because many distributions come with a decent
set of Python packages.

hp
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | ***@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Grant Edwards
2024-05-18 16:48:14 UTC
Permalink
Post by Peter J. Holzer
Post by Gordinator
To be fair, the problem is the fact that they use Windows (but I
guess Linux users have to deal with venvs, so we're even.
I don't think Linux users have to deal with venvs any more than
Windows users. Maybe even less because many distributions come with
a decent set of Python packages.
I've been using Python on Linux almost daily for 25 years, and I've
yet to use a venv...

--
Grant
Mats Wichmann
2024-05-18 18:04:07 UTC
Permalink
Post by Grant Edwards
Post by Peter J. Holzer
Post by Gordinator
To be fair, the problem is the fact that they use Windows (but I
guess Linux users have to deal with venvs, so we're even.
I don't think Linux users have to deal with venvs any more than
Windows users. Maybe even less because many distributions come with
a decent set of Python packages.
I've been using Python on Linux almost daily for 25 years,
same here, but:

and I've
Post by Grant Edwards
yet to use a venv...
Distros have do offer a good selection of packaged Python bits, yes, but
only for the version of Python that's "native" to that distro release.
If you need to test other versions of Python, you're mostly on your own.
Just as an example, for a particular project I had one test machine
running Fedora 38 until just a couple weeks ago, with Python 3.11 as
"native" with a full suite of packages, but I needed to test 3.12 and
then the 3.13 pre-releases, as well as occasionally sanity-check the
"oldest supported Python for this project", which turned out to be 3.6.
I could build all those Pythons myself and install them to a location I
can "python3.xx -m pip install" to, but Fedora is nice enough to package
up a whole bunch of past and future Python versions, so why? And Fedora
really discourages doing installs via pip to a system-packaged Python.
So venvs make managing all that pretty convenient. Dunno why everybody's
so down on venvs...
Piergiorgio Sartor
2024-05-18 18:12:33 UTC
Permalink
On 18/05/2024 20.04, Mats Wichmann wrote:
[...]
Post by Mats Wichmann
So venvs make managing all that pretty convenient. Dunno why everybody's
so down on venvs...
Only people which are *not* using python... :-)

In my experience, venvs is the only possible
way to use python properly.

The dependency nightmare created by python, pip
and all the rest cannot be resolved otherwise.

It seems backward compatibility is a taboo...

bye,
--
piergiorgio
Peter J. Holzer
2024-05-19 06:49:06 UTC
Permalink
Post by Piergiorgio Sartor
Post by Mats Wichmann
So venvs make managing all that pretty convenient. Dunno why everybody's
so down on venvs...
Only people which are *not* using python... :-)
In my experience, venvs is the only possible
way to use python properly.
That's very much depends on what you mean by properly.

Personally, I use venvs a lot. But most of the reasons have more to do
with team culture than technical constraints. In a different situation
(e.g. if all our developers used Linux and preferrably the same version)
I could see myself using venvs much less or maybe not at all.
Post by Piergiorgio Sartor
The dependency nightmare created by python, pip and all the rest
cannot be resolved otherwise.
That's what package management on Linux is for. Sure, it means that you
won't have the newest version of anything and some packages not at all,
but you don't have to care about dependencies. Or updates.

(Missing packages can be a problem: Is there a script to automatically
generate .deb packages from PyPI? I haven't looked recently ...)
Post by Piergiorgio Sartor
It seems backward compatibility is a taboo...
I have recently written a script which checks out the newest version of
the project, creates a fresh venv using a requirements.txt without
version numbers and runs the test suite. If there is any action required
(either because a test fails or because there is a newer version of any
dependent package) it will create a ticket in redmine. Oh, and this
script runs on a staging server which has the same Linux distribution
(and hence the same Python version) as the production server.
Seems to work, but that is only necessary because we are using venvs. If
we relied on the distro's package management that would basically be a
non-issue.

hp
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | ***@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Piergiorgio Sartor
2024-05-19 13:44:57 UTC
Permalink
On 19/05/2024 08.49, Peter J. Holzer wrote:
[...]
Post by Peter J. Holzer
That's what package management on Linux is for. Sure, it means that you
won't have the newest version of anything and some packages not at all,
but you don't have to care about dependencies. Or updates.
Well, that doesn't work as well.
Distributions do not pack everything, this
also depending on licenses.

Sometimes, or often, you need to use the
*latest* version of something, due to some
bugfix or similar.

The distribution does not always keep up
to date everything, so you're stuck.

The only solution is a venv, with all
needed packages for the given task.

Typical problem with PyTorch / TensorFlow.

In case of trouble, the first answer is:
"Check with the latest (nightly) release".

Which means installing something *outside*
the Linux distribution support.
And this impossible, because this will pull
in dependencies like crazy, which are not
(yet) in the Linux distribution path.

Saying it differently, the latest greatest
update is not a wish, it's a must...

So, long story short, the only solution I
know are venvs...

Of course, other solutions are welcome!

bye,
--
piergiorgio
Left Right
2024-05-19 23:15:06 UTC
Permalink
There are several independent problems here:

1. Very short release cycle. This is independent of the Python venv
module but is indirectly influenced by Python's own release cycle.
Package maintainers don't have time for proper testing, they are
encouraged to release a bunch of new (and poorly tested) versions, and
they never get a break. So, when you install the latest, there will be
something else broken. There's never a window to properly test
anything.

2. Python made a very short-sighted decision about how imports work.
Python doesn't have a concept of "application", and therefore there's
no way to specify dependencies per application (and imports import
anything that's available, not versioned). That's why every Python
application ends up carrying its own Python, with the version of its
own dependencies around. Python's venv module is just an
acknowledgement of this design flaw. I.e. the proper solution
would've been a concept of application and per-application dependency
specification, but instead we got this thing that doesn't really work
(esp. when native modules and shared libraries are considered), but it
"works" often enough to be useful.

3. The Python community grew to be very similar to what PHP 4 was,
where there were several "poisonous" examples, which were very popular
on the Web, which popularized a way of working with MySQL databases
that was very conducive to SQL injections. Python has spread very bad
ideas about project management. Similar to how PHP came up with
mysql_real_escape() and mysql_this_time_promise_for_real_escape() and
so on functions, Python came up with bad solutions to the problems
that had to be fixed by removing bad functionality (or, perhaps,
education). So, for example, it's very common to use requirements.txt,
which is generated by running pip freeze (both practices are bad
ideas). Then PyPA came up with a bunch of bad ideas in response to
problems like this, eg. pyproject.toml. In an absurd way very much
mirroring the situation between makefiles and makefiles generated by
autotools, today Python developers are very afraid of doing simple
things when it comes to project infrastructure (it absolutely has to
be a lot of configuration fed into another configuration, processed by
a bunch of programs to generate even more configuration...) And most
Python programmers don't really know how the essential components of
all of this infrastructure work. They rely on a popular / established
pattern of insane multi-step configuration generation to do simple
things. And the tradition thus developed is so strong, that it became
really cultish. This, of course, negatively contributes to the overall
quality of Python packages and tools to work with them.

Unfortunately, the landscape of Python today is very diverse. There's
no universally good solution to package management because it's broken
in the place where nobody is allowed to fix it. Commercial and
non-commercial bodies alike rely on people with a lot of experience
and knowledge of particular Python gotchas to get things done. (Hey,
that's me!) And in different cases, the answer to the problem will be
different. Sometimes venv is good enough. Other times you may want a
container or a vm image. Yet in a different situation you may want a
PyPA or conda package... and there's more.

On Sun, May 19, 2024 at 4:05 PM Piergiorgio Sartor via Python-list
Post by Piergiorgio Sartor
[...]
Post by Peter J. Holzer
That's what package management on Linux is for. Sure, it means that you
won't have the newest version of anything and some packages not at all,
but you don't have to care about dependencies. Or updates.
Well, that doesn't work as well.
Distributions do not pack everything, this
also depending on licenses.
Sometimes, or often, you need to use the
*latest* version of something, due to some
bugfix or similar.
The distribution does not always keep up
to date everything, so you're stuck.
The only solution is a venv, with all
needed packages for the given task.
Typical problem with PyTorch / TensorFlow.
"Check with the latest (nightly) release".
Which means installing something *outside*
the Linux distribution support.
And this impossible, because this will pull
in dependencies like crazy, which are not
(yet) in the Linux distribution path.
Saying it differently, the latest greatest
update is not a wish, it's a must...
So, long story short, the only solution I
know are venvs...
Of course, other solutions are welcome!
bye,
--
piergiorgio
--
https://mail.python.org/mailman/listinfo/python-list
Alan Gauld
2024-05-19 07:32:46 UTC
Permalink
Post by Piergiorgio Sartor
Post by Mats Wichmann
So venvs make managing all that pretty convenient. Dunno why everybody's
so down on venvs...
Not so much down on them, they are just one extra step that's
mostly not needed(in my use case)
Post by Piergiorgio Sartor
Only people which are *not* using python... :-)
In my experience, venvs is the only possible
way to use python properly.
Well, I've been using Python since 1998 on Linux, Windows
and MacOS and have yet to find a use for a venv. I've
played with them when they first came out but haven't
actually found a scenario where I've thought "I need
a venv for that!"

But then I'm a sole user, I have 3 or 4 projects going
but only me working on them. I only have 2 Python versions
at any time and the OS handles that just fine without
any venvs.
Post by Piergiorgio Sartor
The dependency nightmare created by python, pip
and all the rest cannot be resolved otherwise.
I've honestly never experienced this "nightmare".
I install stuff and it just works.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
Gilmeh Serda
2024-05-19 18:13:23 UTC
Permalink
Post by Alan Gauld
I've honestly never experienced this "nightmare".
I install stuff and it just works.
Hear! Hear! Me too! And all that.

I'm on Manjaro, which is a tad finicky about other people touching its
Python since it's used for lots of things. I install things for myself
only.

Was there a reason they chose the name Pip?

From WordNet (r) 3.0 (2006) [wn]:

pip
n 1: a disease of poultry
2: a minor nonspecific ailment
3: a small hard seed found in some fruits
4: a mark on a die or on a playing card (shape depending on the
suit) [syn: {spot}, {pip}]
5: a radar echo displayed so as to show the position of a
reflecting surface [syn: {blip}, {pip}, {radar target}]
v 1: kill by firing a missile [syn: {shoot}, {pip}]
2: hit with a missile from a weapon [syn: {shoot}, {hit}, {pip}]
3: defeat thoroughly; "He mopped up the floor with his
opponents" [syn: {worst}, {pip}, {mop up}, {whip}, {rack up}]
--
Gilmeh

All the evidence concerning the universe has not yet been collected, so
there's still hope.
2***@potatochowder.com
2024-05-19 18:57:05 UTC
Permalink
On 2024-05-19 at 18:13:23 +0000,
Post by Gilmeh Serda
Was there a reason they chose the name Pip?
Package Installer for Python

https://pip.pypa.io/en/stable/index.html

Every time I see PIP, I think Peripheral Interchange Program, but I'm
old.
MRAB
2024-05-19 19:01:20 UTC
Permalink
Post by Gilmeh Serda
Post by Alan Gauld
I've honestly never experienced this "nightmare".
I install stuff and it just works.
Hear! Hear! Me too! And all that.
I'm on Manjaro, which is a tad finicky about other people touching its
Python since it's used for lots of things. I install things for myself
only.
Was there a reason they chose the name Pip?
[snip]
From https://pip.pypa.io/en/stable/:

"pip is the package installer for Python."

It's an acronym.
2***@potatochowder.com
2024-05-19 18:54:41 UTC
Permalink
On 2024-05-19 at 18:13:23 +0000,
Post by Gilmeh Serda
Was there a reason they chose the name Pip?
Package Installer for Python

https://pip.pypa.io/en/stable/index.html
Grant Edwards
2024-05-19 22:33:51 UTC
Permalink
Post by Gilmeh Serda
Post by Alan Gauld
I've honestly never experienced this "nightmare".
I install stuff and it just works.
Hear! Hear! Me too! And all that.
I'm on Manjaro, which is a tad finicky about other people touching its
Python since it's used for lots of things. I install things for myself
only.
Was there a reason they chose the name Pip?
I always assumed it was in honor of the PIP (Peripheral Interchange
Program?) utility that was used to copy files around on CP/M and DEC's
PDP-11 OSes.

--
Grant
Gordinator
2024-05-20 18:49:32 UTC
Permalink
Post by Gilmeh Serda
I'm on Manjaro
Of course, I'm not here to tell you how to use your computer, and it's
great that you're using Linux, but I'd suggest that you look into
installing Arch Linux proper.

Arch Linux isn't as difficult as people make it out to be (I'd argue
that anyone who's had to deal with the Calamares installer has seen
worse), and it's hugely rewarding as it gives you a fundamental overview
of how your system operates and what makes it tick, since you need to
install it yourself manually.

Plus, Manjaro holds back packages by about a month or so, sometimes
more, which breaks AUR packages, which are designed around Arch Linux's
packages, which are newer. On top of this, the Manjaro team just can't
be trusted with basic things like not having their SSL certs expire on
their website (this has happened half a dozen times in the past, which
is embarassing, given that things like certbot make installing a
certificate the easiest thing in the world).

Again, I'm not some power hungry elitist Arch Linux shill or whatever,
it's your computer, use it how you want, these are just my suggestions.
Don't say I didn't warn you though :)
Grant Edwards
2024-05-19 14:09:07 UTC
Permalink
Post by Alan Gauld
Post by Piergiorgio Sartor
The dependency nightmare created by python, pip
and all the rest cannot be resolved otherwise.
I've honestly never experienced this "nightmare".
I install stuff and it just works.
Same here. I occasonlly use pip to install something that isn't
packaged for Gentoo, but it doesn't seem to cause problems — let alone
nightmares. I also occasionally package something myself.

--
Grant
Thomas Passin
2024-05-19 11:51:46 UTC
Permalink
Post by Alan Gauld
[snip]
The dependency nightmare created by python, pip
and all the rest cannot be resolved otherwise.
I've honestly never experienced this "nightmare".
I install stuff and it just works.
One way it can bite even you is if you have a program installed whose
requirements claim it needs a certain library of version no higher than
X, and you already already have a later version of that library
installed since one of your other programs requires it. Pip won't
install the new program because of this conflict.

In reality, you may know the the new program would work fine with the
version of the library you installed, but the packagers of the new
program didn't realize they should have updated their requirements.

With venvs, you can have separate environments for each. Of course this
doesn't help if you need both packages in the same environment...
Akkana Peck
2024-05-20 21:48:12 UTC
Permalink
Post by Alan Gauld
Post by Mats Wichmann
So venvs make managing all that pretty convenient. Dunno why everybody's
so down on venvs...
Not so much down on them, they are just one extra step that's
mostly not needed(in my use case)
Years ago, I used to have trouble with pip install --user on Debian -- sometimes things would end up under .local, sometimes in other places that I've forgotten. So I reluctantly started using venvs.

And you know, they work fine. I have one master venv that I created with
python3 -m venv --system-site-packages ~/pythonenv/envname
and I activate that automatically when I log in, so when I need to install anything that Debian doesn't package, I just pip install it (no --user or --break-system-packages needed) and it installs to that venv.

Every so often I need to regenerate it (like when Debian updates the system Python version) but that's easy to do: I don't try to duplicate what's installed there, I just delete the old venv, create a new one and then pip install packages as needed.

I have a few special venvs (without --system-site-packages) for specific purposes, but most of the time I'm just using my default venv and it's all pretty transparent and automatic.

I know this isn't the usual pythonista model of "you should have a zillion different venvs, one for each program you use, and never use system Python packages", but it works well for me: my pip installed packages are all in a predictable place, and I get security updates for all the software Debian *does* package. That's my biggest beef with pip, the lack of an easy way to update everything at once, and it's the reason I prefer Debian packages when available.

...Akkana
Roel Schroeven
2024-05-21 08:03:09 UTC
Permalink
Post by Akkana Peck
Every so often I need to regenerate it (like when Debian updates the system Python version) but that's easy to do: I don't try to duplicate what's installed there, I just delete the old venv, create a new one and then pip install packages as needed.
I know this isn't the usual pythonista model of "you should have a zillion different venvs, one for each program you use, and never use system Python packages", but it works well for me: my pip installed packages are all in a predictable place, and I get security updates for all the software Debian *does* package. That's my biggest beef with pip, the lack of an easy way to update everything at once, and it's the reason I prefer Debian packages when available.
If you have a requirements.txt file with all packages you want, I think
you can do pip -install --upgrade -r requirements.txt to update them
all. That only works if you don't specify exact versions in the
requirements.txt file, so don't use the output of pip freeze to generate
that requirements file. Just create it yourself: it's a simple text file
with one package per line. Also I prefer not to include dependencies in
it for use cases like this (it's another story for packaging, where it
can be useful or requirements.txt to mirror your exact environment with
dependencies and specific versions).

Having such a requirements.txt file also makes it easier to install all
the packages again after you re-create your venv.
--
"I love science, and it pains me to think that to so many are terrified
of the subject or feel that choosing science means you cannot also
choose compassion, or the arts, or be awed by nature. Science is not
meant to cure us of mystery, but to reinvent and reinvigorate it."
-- Robert Sapolsky
Grant Edwards
2024-05-18 18:41:45 UTC
Permalink
Post by Mats Wichmann
Distros have do offer a good selection of packaged Python bits, yes, but
only for the version of Python that's "native" to that distro release.
If you need to test other versions of Python, you're mostly on your own.
For a few years I needed both 2.x and 3.x installed, but my distro
(Gentoo) handled that fine (I think most others do also). Gentoo also
allows multiple minor versions to be installed (currently I have 3.11
and 3.12 on this machine).

But, since Gentoo is a source-based meta-distro, when I install a
Python package, the package manager knows how to install it for all
installed Python versions that are supported by the package.

I can't think of why I would need a venv unless I needed to test
something with a Python version that isn't available for Gentoo. That
said, there are currently 7 versions available (2.7.18, 3.8.19,
3.9.19, 3.10.14, 3.11.9, 3.12.3, 3.13.0).

3.13 isn't marked stable yet in the Gentoo package database, and I
apparently have some Python app/package installed that doesn't yet
support 3.12, so I've currently got both 3.12 and 3.11.
Peter J. Holzer
2024-05-18 15:08:43 UTC
Permalink
Post by Grant Edwards
Post by Alan Gauld
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly
competent Python user, and I wish to try a new project idea. What
references can I use when writing my terminal emulator? I wish for
it to be a true terminal emulator as well, not just a Tk text
widget or something like that.
The first thing is to decide which terminal.
If you want to make life easier, make it a superset of a terminal that
already exists in the terminfo database.
Going with some sort of ANSI terminal will probably provide
operability even with dumb apps which ignore $TERM and just spit out
basic ANSI escape sequences.
And if you want to go for a superset, xterm might be one of the more
useful: https://www.xfree86.org/current/ctlseqs.html
Post by Grant Edwards
If you really want to break trail, you could invent your own control
sequences, which means you'll have to write terminfo and/or termcap
entries as well as the terminal emulator.
Right. A saner model than ANSI and its supersets might be a good idea
conceptionally. But I'd expect quite a few programs to break.
Post by Grant Edwards
Post by Alan Gauld
A VT100 is very different from a 3270. And even a VT330 is quite
different from a VT100 although sharing a common subset of control
codes. And if you start looking at graphical terminals things get
even more interesting!
"Intersting" is putting it mildly...
Yup. Also there aren't many programs which use, e.g. xterm's
pixel-graphics capabilities.

OTOH, there is something like domterm[1], which can (theoretically)
display anything a browser can display.

hp


[1] https://domterm.org/index.html
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | ***@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Peter J. Holzer
2024-05-18 15:17:18 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can I
use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.
If you have any advice, please do let me know!
true terminal emulator as well, not just a Tk text widget or
something like that
If you want to write a GUI terminal, than that *is* a terminal emulator and
*has* a text widget as its visible core. If you want to write something like
getty which runs on the virtual terminals (Ctrl+Alt+F*) than that is a
terminal (not a terminal emulator).
Getty isn't a terminal (unless there is another program of the same
name). It's a small program to set up a serial communication line.
Basically it waits for the modem to connect, sets the relevant
parameters (bit rate, byte width, parity, ...) and then hands over to
login. Of course in the case of a linux console there is no modem and no
serial line involved, so it doesn't have much to do. (Of course this
raises the question whether the Linux console is a terminal or a
terminal emulator ...)

hp
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | ***@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Piergiorgio Sartor
2024-05-18 15:31:02 UTC
Permalink
Post by Gordinator
I wish to write a terminal emulator in Python. I am a fairly competent
Python user, and I wish to try a new project idea. What references can I
use when writing my terminal emulator? I wish for it to be a true
terminal emulator as well, not just a Tk text widget or something like that.
If you have any advice, please do let me know!
I would start writing down what this
"terminal emulator" should do.

Then, collect information about what
module / library can support the features.

bye,
--
piergiorgio
Barry
2024-05-19 21:45:09 UTC
Permalink
Post by Peter J. Holzer
I don't think Linux users have to deal with venvs
Modern debian (ubuntu) and fedora block users installing using pip.
You must use a venv to pip install packages from pypi now.
This is implemented in python and pip and enabled by the distros.

There are ways to turn off the blocking, but it’s strongly discouraged
by the distros.

Barry
Karsten Hilbert
2024-05-19 22:00:28 UTC
Permalink
Post by Barry
Post by Peter J. Holzer
I don't think Linux users have to deal with venvs
Modern debian (ubuntu) and fedora block users installing using pip.
You must use a venv to pip install packages from pypi now.
Which makes one wonder how one is supposed to package Python
applications requiring modules not yet packaged by Debian.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
Skip Montanaro
2024-05-19 22:08:37 UTC
Permalink
Modern debian (ubuntu) and fedora block users installing using pip.
Even if you're telling it to install in ~/.local? I could see not allowing
to run it as root.

I honestly haven't tried. Maybe I should... 🤔 I have an old laptop running
XUbuntu 22.04 which I generally only use to compile the most recent
branches on GitHub (main, 3.12, & 3.13 at the moment).

Skip
Roel Schroeven
2024-05-19 22:26:03 UTC
Permalink
Post by Barry
Modern debian (ubuntu) and fedora block users installing using pip.
Even if you're telling it to install in ~/.local? I could see not allowing
to run it as root.
I assumed pip install --user would work, but no. I tried it (on Debian
Post by Barry
$ pip install --user docopt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at
the risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.
Exactly the same output for sudo pip install.

For easy reference here's a link to that PEP 668:
https://peps.python.org/pep-0668/
Which links to the "Externally Managed Environments" on the PyPA specs
page:
https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments
--
"If you don't read the newspaper, you're uninformed. If you read the newspaper,
you're mis-informed."
-― Onbekend (dikwijls toegeschreven aan Mark Twain, waarschijnlijk onterecht)
Grant Edwards
2024-05-19 22:34:34 UTC
Permalink
Post by Barry
Post by Peter J. Holzer
I don't think Linux users have to deal with venvs
Modern debian (ubuntu) and fedora block users installing using pip.
You can't even use pip to do "user" installs?

--
Grant
Thomas Passin
2024-05-19 22:49:56 UTC
Permalink
Post by Barry
Modern debian (ubuntu) and fedora block users installing using pip.
Even if you're telling it to install in ~/.local? I could see not allowing
to run it as root.
I honestly haven't tried. Maybe I should... 🤔 I have an old laptop running
XUbuntu 22.04 which I generally only use to compile the most recent
branches on GitHub (main, 3.12, & 3.13 at the moment).
On some (maybe all) of my Linux VMs, pip - the one installed for the
system - won't install something even with --user unless you use the
--break-system-packages flag.
Thomas Passin
2024-05-19 23:35:13 UTC
Permalink
Post by Grant Edwards
Post by Barry
Post by Peter J. Holzer
I don't think Linux users have to deal with venvs
Modern debian (ubuntu) and fedora block users installing using pip.
You can't even use pip to do "user" installs?
Nope, often not. The error messages I've gotten do tell you to use the
"--break-system-packages" flag if you really, really want to install the
package. So

python3 -m pip install --user --break-system-packages [--upgrade]
<package name>

Or install an additional version of Python that isn't managed by the system.
Post by Grant Edwards
Grant
Thomas Passin
2024-05-19 22:28:05 UTC
Permalink
Post by Karsten Hilbert
Post by Barry
Post by Peter J. Holzer
I don't think Linux users have to deal with venvs
Modern debian (ubuntu) and fedora block users installing using pip.
You must use a venv to pip install packages from pypi now.
Which makes one wonder how one is supposed to package Python
applications requiring modules not yet packaged by Debian.
I confess, I sometimes install those packages using pip's self-warning
option "--break-system-packages". Naturally I only install them with
--user. Another option besides venvs is to install a different version
of Python from python.org, not by using the system installer. For
example, if your system version is, say, 3.8.10 (as it is on my Mint
VM), install 3.11.9 and make sure you always launch pip with

python3.11 -m pip ....

I have done this on some VMs, and use the system's Python on some
others. Happily none of my Linux VMs are critical for anything. I can
blow them away and recreate them without worry.
Peter J. Holzer
2024-05-20 09:58:59 UTC
Permalink
Post by Skip Montanaro
Post by Barry
Modern debian (ubuntu) and fedora block users installing using pip.
Even if you're telling it to install in ~/.local? I could see not allowing
to run it as root.
I assumed pip install --user would work, but no. I tried it (on Debian 12
Post by Skip Montanaro
$ pip install --user docopt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at the
risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.
Exactly the same output for sudo pip install.
This message (quoted in all its glory) is too long to be useful. The
Post by Skip Montanaro
You can override this, at the risk of breaking your Python
installation or OS, by passing --break-system-packages.
(I admit I didn't see this the first time I got this message)

python3 -m pip install --user --break-system-packages <packagename>
does indeed install into ~/.local/lib/python3.XX/site-packages.

This inconvenient, but otoh I have accidentally installed packages into
~/.local in the past, so maybe it's good to make that more explicit.

hp
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | ***@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Gordinator
2024-05-20 18:43:08 UTC
Permalink
Post by Peter J. Holzer
Post by Skip Montanaro
Post by Barry
Modern debian (ubuntu) and fedora block users installing using pip.
Even if you're telling it to install in ~/.local? I could see not allowing
to run it as root.
I assumed pip install --user would work, but no. I tried it (on Debian 12
Post by Skip Montanaro
$ pip install --user docopt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at the
risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.
Exactly the same output for sudo pip install.
This message (quoted in all its glory) is too long to be useful. The
Post by Skip Montanaro
You can override this, at the risk of breaking your Python
installation or OS, by passing --break-system-packages.
(I admit I didn't see this the first time I got this message)
python3 -m pip install --user --break-system-packages <packagename>
does indeed install into ~/.local/lib/python3.XX/site-packages.
This inconvenient, but otoh I have accidentally installed packages into
~/.local in the past, so maybe it's good to make that more explicit.
hp
Perhaps an alias like so:

$ alias 'pip install'='pip install --user --break-system-packages'

Would work here? Someone please advise if that is a good idea.
Loading...