Trey Hunner - Python

Python Morsels now has spaced repetition

2026-09-03T17:30:00-07:00

Nearly every book I’ve read on teaching and education over the past decade has talked about the value of spaced repetition. For much of that time, spaced repetition has been something I would recommend learners do, but it wasn’t something I helped anyone do… until now.

Python Morsels now has Daily Recall: a spaced repetition system for Python programmers.

We learn by recalling, not by reading

The most effective learning techniques all rely on active recall: trying to remember something without looking it up, whether with flash cards, by explaining an idea in your own words, or by doing a task that requires it.

We don’t learn by putting information into our heads. We learn by retrieving information from our heads. That’s why Python Morsels has always been built around exercises rather than videos: writing code is the most useful form of recall for a Python programmer.

But not everything worth remembering warrants an entire Python exercise.

Which uv command runs a tool without installing it?
What is the time complexity of various list operations?
Which string method can remove a substring from the end of a string?

A 20-minute exercise is overkill for practicing something that small. But practicing it just once isn’t enough either. A quick question, asked again just before you’d forget, is a much better fit.

Spaced repetition beats the forgetting curve

Many of the things I learned in school are long gone, especially the ones I haven’t thought about even once in years. I think I could explain photosynthesis in 9th grade. I can’t today.

This is explained by the forgetting curve: we forget what we don’t recall, and the rate of forgetting is somewhat predictable. Spaced repetition is about using active recall to beat the forgetting curve. Instead of recalling an idea over and over right after learning it, you wait until it has started to fade, and then try to recall it. Each successful recall earns a longer wait before the next one: minutes at first, then hours, then days, and eventually weeks and months.

The tricky part is the timing: when should you try to recall each thing? That’s where an algorithm helps. A spaced repetition system tracks every idea you’re trying to remember and prompts you to recall each one right before you’d forget it.

What Daily Recall does

Back in April, just before Earth Day, I made Whereabouts.Earth to help me learn the name and location of every country in the world. When I started, I could name about 90 of the 197 countries on a map. By mid-June, with about 10 minutes of practice a day, I knew all of them.

Daily Recall is the same idea, but for Python. You pick the packs you want to practice, and each day it asks you a few questions from them. Answer a question correctly and it’ll be a while before you see that one again. Miss it and you’ll see it again soon.

Daily Recall uses FSRS for scheduling, which is one of the most effective spaced repetition algorithms (it’s an option in the Anki flash card app).

If you have trouble remembering which operations on different data structures are fast and which are slow, there’s a Time Complexity pack for that. If you’re struggling to remember the many different subcommands that uv supports, there’s a uv pack for that. There are also packs on string methods, built-in functions, dictionaries, iterable unpacking, f-strings, pytest, and what’s new in Python 3.13 and 3.14. I’m hoping to release about one new pack each week over the next many months.

Daily Recall also works well on a phone because recall questions don’t require typing a bunch of code. So you can replace 5 minutes of your daily doomscrolling with 5 minutes of extra Python learning.

Early users who practiced about 5 minutes a day ended their first month with 25 to 50 new things they could still recall weeks after last seeing them.

You can use Daily Recall for free

Most Daily Recall packs are free, and the rest are included with the All Access plan. To get started, create a free Python Morsels account, pick a pack or two, and answer a few questions.

And whether or not you ever use Daily Recall, I’d recommend spaced repetition. If you’d rather write your own flash cards, on Python or anything else, Anki works well too. The next time you learn something new in Python, don’t just read it a second time. Try to recall it tomorrow, and then again next week.

Try Daily Recall

Reorganizing Python's sys module

2026-08-12T17:30:00-07:00

The sys module is the primary junk drawer of the Python standard library.

A good junk drawer holds miscellaneous items that don’t have another sensible home.

I often think of utils modules as “junk drawer” modules. I believe that both utils modules and junk drawers have their purpose, but junk drawers can get out of hand.

As I recently noted in my talk about pathlib, I see both the sys module and the os module as junk drawer modules. They serve a similar purpose to utils modules, but they have a different name.

In this post, I wonder: what would Python’s sys be like if it was designed today, from scratch?

The overview: 9 new `sys` submodules

As of Python 3.15, the sys module has 115-121 attributes (depending on whether you’re in the REPL and whether an exception has occurred), and 63 of those are functions.

All of those names need permanent homes.

How can we reorganize a utils-style module that’s grown quite large?

Give it submodules!

No, really… this isn’t the worst solution. Django’s utils package isn’t so bad.

So we could split sys into these 9 submodules:

sys.cli: for command-line argument handling and program control
sys.imports: related to imports and modules
sys.io: handles standard I/O streams
sys.repl: handles REPL display control
sys.interpreter: system information and installation details
sys.memory: memory management tools used for profiling, optimization, and debugging
sys.exceptions: exception handling
sys.profile: profiling and introspection
sys.runtime: interpreter runtime behavior

You may notice some similarities to other modules in the Python standard library. That’s a hint that it might be worth moving some of these utilities into other parts of the standard library. But moving functionality between top-level modules is a much bigger change, so let’s set that idea aside.

For now, let’s take a closer look at our tentatively re-organized sys package.

The `sys` package and its submodules

A couple of these 9 submodules have only a handful of attributes, a couple have over 20, and the rest fall somewhere in the middle.

`sys.cli`

This submodule would handle command-line arguments and program control:

argv: Command-line arguments list
orig_argv: Original unmodified arguments
exit(): Function to terminate Python
flags: Named tuple of interpreter flags
_xoptions: Dictionary of -X command options

`sys.imports`

Everything related to imports and modules:

path: Module search path list
modules: Dictionary of loaded modules
builtin_module_names: Tuple of built-in modules
stdlib_module_names: Frozen set of standard library modules
path_hooks: List of path-to-finder callables
path_importer_cache: Finder object cache
meta_path: Meta path finder objects
pycache_prefix: Bytecode cache directory
set_lazy_imports(), get_lazy_imports(), set_lazy_imports_filter(), get_lazy_imports_filter(), lazy_modules: Lazy import controls (Python 3.15+)

`sys.io`

The standard I/O streams:

stdin: Standard input stream
stdout: Standard output stream
stderr: Standard error stream
__stdin__, __stdout__, __stderr__: The original values of those three streams (useful for restoring them after replacing them)

`sys.repl`

Hooks and settings for Python’s interactive prompt:

displayhook(): Called to show the result of each REPL expression
__displayhook__: The original value of displayhook
ps1: The primary prompt string (>>>)
ps2: The continuation prompt string (...)
__interactivehook__: Called when an interactive session starts up
_baserepl(): Starts the basic fallback REPL

`sys.interpreter`

Information about the Python build, the Python installation, and the operating system:

platform: Platform identifier string (linux, darwin, win32, etc.)
version: Python version string
version_info: Python version as a named tuple
implementation: Python implementation details (CPython, PyPy, etc.)
executable: Path to the Python interpreter
prefix, exec_prefix: Installation prefixes
base_prefix, base_exec_prefix: Installation prefixes, ignoring virtual environments
platlibdir: Platform-specific library directory name
maxunicode: Maximum Unicode code point (1114111)
maxsize: Maximum size of containers
byteorder: Native byte order ('little' or 'big')
hexversion: Version encoded as single integer
api_version: C API version
copyright: Python copyright notice
abiflags: ABI flags from PEP 3149
abi_info: ABI details namespace (Python 3.15+)
float_info: Floating point implementation details
int_info: Integer implementation details
hash_info: Hash algorithm parameters
float_repr_style: float repr style ('short' or 'legacy')
winver, dllhandle, getwindowsversion(): Windows-specific details
getandroidapilevel(): Android-specific detail
_base_executable, _framework, _git, _home, _stdlib_dir: Assorted build and installation details

`sys.memory`

Memory management tools used for profiling, optimization, and debugging:

getsizeof(): Size of an object in bytes
getrefcount(): Number of references to an object
getallocatedblocks(): Number of allocated memory blocks
getunicodeinternedsize(): Number of interned strings
intern(): Intern a string
_is_interned(): Check whether a string is interned
_is_immortal(): Check whether an object is immortal
_debugmallocstats(): Print memory allocator statistics
_clear_type_cache(), _clear_internal_caches(): Clear interpreter caches

`sys.exceptions`

Tools for accessing and handling exceptions:

exc_info(): Currently handled exception, as a 3-tuple
exception(): Currently handled exception (Python 3.11+)
last_exc, last_type, last_value, last_traceback: The most recent unhandled exception, mostly for REPL use
excepthook(): Called to display unhandled exceptions
__excepthook__: The original value of excepthook
unraisablehook(): Called for exceptions that can’t be raised
__unraisablehook__: The original value of unraisablehook
tracebacklimit: Maximum number of traceback levels to display

`sys.profile`

Profiling, tracing, auditing, and other runtime introspection:

setprofile(), getprofile(): Profiling hooks
_setprofileallthreads(): Set profile function for all threads (Python 3.12+)
settrace(), gettrace(): Tracing hooks
_settraceallthreads(): Set trace function for all threads (Python 3.12+)
call_tracing(): Call a function with tracing enabled
monitoring: Low-overhead monitoring events namespace (Python 3.12+)
_getframe(), _getframemodulename(): Frame inspection
_current_frames(), _current_exceptions(): Frames and exceptions across all threads
activate_stack_trampoline(), deactivate_stack_trampoline(), is_stack_trampoline_active(): Support for the perf profiler (Python 3.12+)
audit(): Raise an auditing event
addaudithook(): Register an audit hook
remote_exec(), is_remote_debug_enabled(): Remote debugging support (Python 3.14+)

`sys.runtime`

Settings that control interpreter runtime behavior:

setrecursionlimit(), getrecursionlimit(): Recursion depth control
setswitchinterval(), getswitchinterval(): Thread switching control
is_finalizing(): Whether the interpreter is shutting down
breakpointhook(): Called by the built-in breakpoint function
__breakpointhook__: The original value of breakpointhook
dont_write_bytecode: Suppress .pyc file generation
warnoptions: Warning filter settings
getfilesystemencoding(), getfilesystemencodeerrors(): Filesystem encoding details
getdefaultencoding(): Default string encoding (always utf-8)
get_int_max_str_digits(), set_int_max_str_digits(): Limit on int-to-string conversion (Python 3.11+)
setdlopenflags(), getdlopenflags(): Dynamic loading control
thread_info: Thread implementation details
_is_gil_enabled(): Whether the GIL is enabled (Python 3.13+)
_jit: JIT compiler introspection namespace
_dump_tracelets(): Dump the JIT’s internal tracelets
_get_cpu_count_config(): Configured CPU count override
get_asyncgen_hooks(), set_asyncgen_hooks(): Async generator lifecycle hooks
get_coroutine_origin_tracking_depth(), set_coroutine_origin_tracking_depth(): Coroutine debugging depth
_enablelegacywindowsfsencoding(): Windows mbcs encoding compatibility

Could this actually be done?

When I first pondered this experiment last year, this was a very hypothetical thought experiment that I assumed could never be done. I still mostly feel the same way.

There are a few big problems with such a refactoring:

What would the transition period look like for the huge amount of code that currently uses existing sys features?
Would backwards compatibility be maintained forever? If so, would this cause more confusion than it’s worth?
How would code that monkey patches attributes like sys.stdout work?

I thought the third question was the biggest roadblock, but I now think it’s the first 2 questions.

Those first 2 questions are big questions and I haven’t thoroughly thought through the upsides and downsides of such a refactoring.

That third question is a technical one, but I think I can answer it… but the answer is messy.

The magic of module-level `setattr`

When Python users want to capture all output from their program to a file, they reassign sys.stdout to an in-memory file-like object. That’s how the contextlib.redirect_stdout helper works, and many testing tools use the same technique.

This monkey patching of sys.stdout is somewhat common, which poses a bit of a problem for us.

Imagine that stdout actually lived in a sys.io submodule. If sys.stdout and sys.io.stdout were two separate attributes, code that assigned to one would be invisible to code that read from the other.

We need a way to synchronize reads and writes of the old flat sys namespace to forward them to the newly nested namespace within the right submodule.

Python has supported customizing module-level attribute reads since Python 3.7, thanks to module-level __getattr__ functions (PEP 562). But Python doesn’t support module-level __setattr__ functions (that idea was proposed in PEP 726 and rejected).

Although… every Python module is an instance of ModuleType, and Python allows changing the class of a module object. If we swap in a ModuleType subclass, we can define whatever __getattr__ and __setattr__ behavior we’d like.

This trick is demonstrated in a proof-of-concept newsys package.

The newsys package reimplements sys as a package with 9 submodules. As a proof of concept, this module simply proxies to the sys module. All reads and writes of newsys.io.stdout or newsys.stdout proxy to the original sys.stdout (since that’s what everything else still uses under the hood in the existing Python interpreter).

What would the transition look like?

If this transition was ever actually done, I imagine it might look something like this:

Add the submodules, with every old flat name still working via forwarding
Update the documentation to nudge folks toward the new names
Soft deprecate the flat names someday (or maybe never)
(Likely never) hard deprecate the flat names

There’s a tiny bit of precedent for sys submodules: sys.monitoring (added in Python 3.12) is an actual module that lives under sys. But since sys isn’t a package, import sys.monitoring doesn’t work, as noted at the top of the sys.monitoring documentation page.

But, I doubt this will ever be done. Python isn’t known for reorganizing modules just to clean things up (outside of the big Python 2/3 split).

Removing names like sys.path and sys.argv would break a huge amount of code… and I can imagine Python tutorials and long-time Python users dragging their feet on re-learning “the new way”. After all… why re-learn something when the old version already works and isn’t going anywhere?

If this was ever done, the flat names might need to keep working forever, and permanent aliases might cause more confusion than such a reorganization is worth.

A thought experiment, not a proposal

I’m not seriously proposing that we actually reorganize sys… at least not seriously enough to draft a PEP.

But I do think there’s a practical takeaway here for our own code. When a utils module grows out of hand, submodules can help. And if other code relies on the old flat names, a module-level __getattr__ function (or that hacky __class__ trick) can keep the old names working while you reorganize.

I doubt sys will ever change, but I had fun imagining a version of Python where it did.

Using a ~/.pdbrc file to customize the Python Debugger

2026-04-06T07:30:00-07:00

Did you know that you can customize the Python debugger (PDB) by creating custom aliases within a .pdbrc file in your home directory or Python’s current working directory?

I recently learned this and I’d like to share a few helpful aliases that I now have access to in my PDB sessions thanks to my new ~/.pdbrc file.

The aliases in my `~/.pdbrc` file

Here’s my new ~/.pdbrc file:

# Custom PDB aliases

# dir obj: print non-dunder attributes and methods
alias dir print(*(f"%1.{n} = {v!r}" for n, v in __import__('inspect').getmembers(%1) if not n.startswith("__")), sep="\n")

# attrs obj: print non-dunder data attributes
alias attrs import inspect as __i ;; print(*(f"%1.{n} = {v!r}" for n, v in __i.getmembers(%1, lambda v: not __i.isroutine(v)) if not n.startswith("__")), sep="\n") ;; del __i

# vars obj: print instance variables (object must have __dict__)
alias vars print(*(f"%1.{k} = {v!r}" for k, v in vars(%1).items()), sep="\n")

# src obj: print source file, line number, and code where a class/function was defined
alias src import inspect as __i;; print(f"{__i.getsourcefile(%1)} on line {__i.getsourcelines(%1)[1]}:\n{''.join(__i.getsource(%1))}") ;; del __i

# loc: print local variables from current frame
alias loc print(*(f"{name} = {value!r}" for name, value in vars().items() if not name.startswith("__")), sep="\n")

This allows me to use:

dir, attrs, and vars to inspect Python objects
src to see the source code of a particular function/class
loc to see the local variables

You might wonder “Can’t I use dir(x) instead of dir x and vars(x) instead of vars x and locals() instead of loc?”

You can!… but those aliases print things out in a nicer format.

A demo of these 5 aliases

Let’s use -m pdb -m calendar to launch Python’s calendar module from the command line while dropping into PDB immediately:

$ python -m pdb -m calendar
> /home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/calendar.py(1)<module>()
-> """Calendar printing functions
(Pdb)

Then we’ll set a breakpoint after lots of stuff has been defined and continue to that breakpoint:

(Pdb) b 797
Breakpoint 1 at /home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/calendar.py:797
(Pdb) c
> /home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/calendar.py(797)<module>()
-> firstweekday = c.getfirstweekday
(Pdb) l
792
793
794     # Support for old module level interface
795     c = TextCalendar()
796
797 B-> firstweekday = c.getfirstweekday
798
799     def setfirstweekday(firstweekday):
800         if not MONDAY <= firstweekday <= SUNDAY:
801             raise IllegalWeekdayError(firstweekday)
802         c.firstweekday = firstweekday

The string representation for that c variable doesn’t tell us much:

(Pdb) !c
<__main__.TextCalendar object at 0x7416de93af90>

If we use the dir alias, we’ll see every attribute that’s accessible on c printed in a pretty friendly format:

(Pdb) dir c
c._firstweekday = 0
c.firstweekday = 0
c.formatday = <bound method TextCalendar.formatday of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatmonth = <bound method TextCalendar.formatmonth of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatmonthname = <bound method TextCalendar.formatmonthname of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatweek = <bound method TextCalendar.formatweek of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatweekday = <bound method TextCalendar.formatweekday of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatweekheader = <bound method TextCalendar.formatweekheader of <__main__.TextCalendar object at 0x7416de93af90>>
c.formatyear = <bound method TextCalendar.formatyear of <__main__.TextCalendar object at 0x7416de93af90>>
c.getfirstweekday = <bound method Calendar.getfirstweekday of <__main__.TextCalendar object at 0x7416de93af90>>
c.itermonthdates = <bound method Calendar.itermonthdates of <__main__.TextCalendar object at 0x7416de93af90>>
c.itermonthdays = <bound method Calendar.itermonthdays of <__main__.TextCalendar object at 0x7416de93af90>>
c.itermonthdays2 = <bound method Calendar.itermonthdays2 of <__main__.TextCalendar object at 0x7416de93af90>>
c.itermonthdays3 = <bound method Calendar.itermonthdays3 of <__main__.TextCalendar object at 0x7416de93af90>>
c.itermonthdays4 = <bound method Calendar.itermonthdays4 of <__main__.TextCalendar object at 0x7416de93af90>>
c.iterweekdays = <bound method Calendar.iterweekdays of <__main__.TextCalendar object at 0x7416de93af90>>
c.monthdatescalendar = <bound method Calendar.monthdatescalendar of <__main__.TextCalendar object at 0x7416de93af90>>
c.monthdays2calendar = <bound method Calendar.monthdays2calendar of <__main__.TextCalendar object at 0x7416de93af90>>
c.monthdayscalendar = <bound method Calendar.monthdayscalendar of <__main__.TextCalendar object at 0x7416de93af90>>
c.prmonth = <bound method TextCalendar.prmonth of <__main__.TextCalendar object at 0x7416de93af90>>
c.prweek = <bound method TextCalendar.prweek of <__main__.TextCalendar object at 0x7416de93af90>>
c.pryear = <bound method TextCalendar.pryear of <__main__.TextCalendar object at 0x7416de93af90>>
c.setfirstweekday = <bound method Calendar.setfirstweekday of <__main__.TextCalendar object at 0x7416de93af90>>
c.yeardatescalendar = <bound method Calendar.yeardatescalendar of <__main__.TextCalendar object at 0x7416de93af90>>
c.yeardays2calendar = <bound method Calendar.yeardays2calendar of <__main__.TextCalendar object at 0x7416de93af90>>
c.yeardayscalendar = <bound method Calendar.yeardayscalendar of <__main__.TextCalendar object at 0x7416de93af90>>

If we use attrs we’ll see just the non-method attributes:

(Pdb) attrs c
c._firstweekday = 0
c.firstweekday = 0

And vars will show us just the attributes that live as proper instance attributes in that object’s __dict__ dictionary:

(Pdb) vars c
c._firstweekday = 0

The src alias can be used to see the source code for a given method:

(Pdb) src c.prmonth
/home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/calendar.py on line 404:
    def prmonth(self, theyear, themonth, w=0, l=0):
        """
        Print a month's calendar.
        """
        print(self.formatmonth(theyear, themonth, w, l), end='')

And the loc alias will show us all the local variables defined in the current scope:

(Pdb) loc
APRIL = __main__.APRIL
AUGUST = __main__.AUGUST
Calendar = <class '__main__.Calendar'>
DECEMBER = __main__.DECEMBER
Day = <enum 'Day'>
FEBRUARY = __main__.FEBRUARY
FRIDAY = __main__.FRIDAY
HTMLCalendar = <class '__main__.HTMLCalendar'>
IllegalMonthError = <class '__main__.IllegalMonthError'>
IllegalWeekdayError = <class '__main__.IllegalWeekdayError'>
IntEnum = <enum 'IntEnum'>
JANUARY = __main__.JANUARY
JULY = __main__.JULY
JUNE = __main__.JUNE
LocaleHTMLCalendar = <class '__main__.LocaleHTMLCalendar'>
LocaleTextCalendar = <class '__main__.LocaleTextCalendar'>
MARCH = __main__.MARCH
MAY = __main__.MAY
MONDAY = __main__.MONDAY
Month = <enum 'Month'>
NOVEMBER = __main__.NOVEMBER
OCTOBER = __main__.OCTOBER
SATURDAY = __main__.SATURDAY
SEPTEMBER = __main__.SEPTEMBER
SUNDAY = __main__.SUNDAY
THURSDAY = __main__.THURSDAY
TUESDAY = __main__.TUESDAY
TextCalendar = <class '__main__.TextCalendar'>
WEDNESDAY = __main__.WEDNESDAY
_CLIDemoCalendar = <class '__main__._CLIDemoCalendar'>
_CLIDemoLocaleCalendar = <class '__main__._CLIDemoLocaleCalendar'>
_get_default_locale = <function _get_default_locale at 0x7416de7c0930>
_locale = <module 'locale' from '/home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/locale.py'>
_localized_day = <class '__main__._localized_day'>
_localized_month = <class '__main__._localized_month'>
_monthlen = <function _monthlen at 0x7416de7c0720>
_nextmonth = <function _nextmonth at 0x7416de7c0880>
_prevmonth = <function _prevmonth at 0x7416de7c07d0>
_validate_month = <function _validate_month at 0x7416de7c05c0>
c = <__main__.TextCalendar object at 0x7416de93af90>
datetime = <module 'datetime' from '/home/trey/.local/share/uv/python/cpython-3.15.0a3-linux-x86_64-gnu/lib/python3.15/datetime.py'>
day_abbr = <__main__._localized_day object at 0x7416de941090>
day_name = <__main__._localized_day object at 0x7416de93acf0>
different_locale = <class '__main__.different_locale'>
error = <class 'ValueError'>
global_enum = <function global_enum at 0x7416dee17480>
isleap = <function isleap at 0x7416dea92770>
leapdays = <function leapdays at 0x7416de7c0460>
mdays = [0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]
month_abbr = <__main__._localized_month object at 0x7416de9411d0>
month_name = <__main__._localized_month object at 0x7416de93ae40>
monthrange = <function monthrange at 0x7416de7c0670>
repeat = <class 'itertools.repeat'>
standalone_month_abbr = <__main__._localized_month object at 0x7416de954fc0>
standalone_month_name = <__main__._localized_month object at 0x7416de941310>
sys = <module 'sys' (built-in)>
weekday = <function weekday at 0x7416de7c0510>

`~/.pdbrc` isn’t as powerful as `PYTHONSTARTUP`

I also have a custom PYTHONSTARTUP file, which is launched every time I start a new Python REPL (see Handy Python REPL modifications). A PYTHONSTARTUP file is just Python code, which makes it easy to customize.

A ~/.pdbrc file is not Python code… it’s a very limited custom file format.

You may notice that every alias line defined in my ~/.pdbrc file is a bunch of code shoved all on one line. That’s because there’s no way to define an alias over multiple lines.

Also any variables assigned in an alias will leak into the surrounding scope… so I have a del statement in a couple of those aliases to clean up a stray variable assignment (from an import).

See the documentation on alias and the top of the debugger commands for more on how ~/.pdbrc files work.

Python Black Friday & Cyber Monday sales (2025)

2025-11-24T09:00:00-07:00

It’s time for some discounted Python-related skill-building. This is my eighth annual compilation of Python learning-related Black Friday & Cyber Monday deals. If you find a Python-related deal in the next week that isn’t on this list, please contact me.

Python-related sales

It’s not even Black Friday yet, but most of these Python-related Black Friday sales are already live:

Python Morsels: I’m offering lifetime access for the second time ever (more details below)
Data School: a new subscription to access all of Kevin’s 7 courses plus all upcoming courses
Talk Python: AI Python bundle, the Everything Bundle, and Michael’s Talk Python in Production
Reuven Lerner: get 20% off your first year of the LernerPython+data tier (code BF2025)
Brian Okken: get 50% off his pytest books and courses (code SAVE50)
Rodrigo: get 50% off all his books including his all books bundle (code BF202550)
Mike Driscoll: get 50% off all his Python books and courses (code BLACKISBACK)
The Python Coding Place: get 50% the all course bundle of 12 courses
Sundeep Agarwal: ~55% off Sundeep’s all books bundle with code FestiveOffer
Nodeledge: get 20% off your first payment (code BF2025)
O’Reilly Media: 40% off the first year with code CYBERWEEK25 ($299 instead of $499)
Manning is offering 50% off from Nov 25 to Dec 1
Wizard Zines: 50% all of Julia Evan’s great zines on various tech topics that I personally find both fun and useful (not Python-related, but one of my favorite annual sales)… this is a one-day Black Friday exclusive sale

I will be keeping an idea on other potential sales and updating this post as I find them. If you’ve seen a sale that I haven’t, please contact me or comment below.

Django-related sales

Adam Johnson has also published a list of Django-related deals for Black Friday (which he’s been doing for a few years now). I’ve included links below to the different sections in Adam’s post…

Courses and books:

Adam’s books: his books and bundles on Django, Git, and GitHub DX and Django test performance are all 50% off
LearnDjango.com course bundle: Will Vincent is offering 50% off his 3 courses
Django 5 By Example: 30% off this book by Antonio Melé
Django in Action: 50% off this book by Christopher Trudeau
Async Patterns in Django: 50% off this book by Paul Bailey

Packages related to Django:

Aidas Bendoraitis: has 3 Django packages available at a 20% discount
Appliku: 30% of annual plans for this Django deployment tool
Saas Pegasus: 50% off the unlimited license for this configurable Django project template

Adam also gives a shout out to sponsoring Django itself.

More on my sale: Python Morsels Lifetime Access

Python Morsels is a hands-on, exercise-driven Python skill-building platform designed to help developers write cleaner, more idiomatic Python through real-world practice. This growing library of exercises, videos, and courses is aimed primarily at intermediate and professional developers. If you haven’t used Python Morsels, you can read about it in my sale announcement post.

This Black Friday / Cyber Monday, I’m offering lifetime access: all current and future content for in a single payment. If you’d like to build confidence in your everyday Python skills, consider this sale. It’s about 50% cheaper than paying annually for five years.

Get lifetime access to Python Morsels

Python Morsels Lifetime Access Sale

2025-11-21T14:42:36-08:00

If you code in Python regularly, you’re already learning new things everyday.

You hit a wall, or something breaks. Then you search around, spend some hours on Stack Overflow, and eventually, you figure it out.

But this kind of learning is unstructured. It’s reactive, instead of intentional.

You fix the problem at hand, but the underlying gaps in your knowledge remain unaddressed.

A more structured way to improve your Python skills

Python Morsels gives you a structured, hands-on way to improve your Python skills through weekly practice:

Notice significant progress with as little as 30 minutes a week
Learn to naturally think more Pythonically
Explore new approaches to problem-solving
Challenge yourself to get outside your comfort zone regularly

Python Morsels is a subscription service because I’m adding new learning resources almost every week.

But through December 1st, you can get lifetime access for a one-time payment.

How it works

When you sign up for Python Morsels, you’ll choose your current Python skill level, from novice to advanced.

Based on your skill level, each Monday I’ll send you a personalized routine with:

a short screencast to watch (or read)
a multi-part exercise to move you outside your comfort zone
a mini exercise that you can accomplish in just 10 minutes
links to dive deeper into subsequent screencasts and exercises

Think of Python Morsels as a gym for your Python skills: you come in for quick training sessions, put in the reps, and make a little more progress each time.

All these resources are accessible to you forever, and with lifetime access you’ll never pay another subscription fee.

What Python Morsels includes

Python Morsels has grown a lot over the past 8 years. Currently, Python Morsels has:

235+ Video Lessons (or screencasts, as I call them)
262+ Hands-On Exercises (with solution walkthroughs)
500+ Optional Bonuses (to challenge every skill level)
303+ Articles Organized Topic-Wise (if you prefer reading)

I’ll be sending you personalized recommendations every week, but you can use these resources however they fit your routine: as learning guides, hands-on practice sessions, quick cheatsheets, long-term reference material, or quick Python workouts.

In addition to this, Python Morsels also gives you access to:

Python Jumpstart - a structured course for beginners
Additional Deep-Dive Courses - structured tracks to master a concept
Exercise Paths - topic-based exercises to strengthen a specific skill

Because Python Morsels runs as an active subscription service, I’m always adding new screencasts, new exercises, and updated material on a weekly or monthly cycle. I also keep everything up-to-date with each new Python release, incorporating newly added features and retiring end-of-life’d Python versions.

Lock in lifetime access

Python Morsels usually costs $240/year but you can get lifetime access through December 1st for a one-time payment. I’ve only offered lifetime access once before in 8 years.

If you’ve been on the fence about subscribing to Python Morsels or want to invest in building a daily learning habit, this is a good time to do it.

Get lifetime access to Python Morsels

If you have questions about the sale, please comment below or email me.

Handy Python REPL Modifications

2025-10-08T19:59:20-07:00

I find myself in the Python REPL a lot.

I open up the REPL to play with an idea, to use Python as a calculator or quick and dirty text parsing tool, to record a screencast, to come up with a code example for an article, and (most importantly for me) to teach Python. My Python courses and workshops are based largely around writing code together to guess how something works, try it out, and repeat.

As I’ve written about before, you can add custom keyboard shortcuts to the new Python REPL (since 3.13) and customize the REPL syntax highlighting (since 3.14). If you spend time in the Python REPL and wish it behaved a little more like your favorite editor, these tricks can come in handy.

I have added custom keyboard shortcuts to my REPL and other modifications to help me more quickly write and edit code in my REPL. I’d like to share some of the modifications that I’ve found helpful in my own Python REPL.

Creating a PYTHONSTARTUP file

If you want to run Python code every time an interactive prompt (a REPL) starts, you can make a PYTHONSTARTUP file.

When Python launches an interactive prompt, it checks for a PYTHONSTARTUP environment variable. If it finds one, it treats it as a filename that contains Python code and it runs all the code in that file, as if you had copy-pasted the code into the REPL.

So all of the modifications I have made to my Python REPL rely on this PYTHONSTARTUP variable in my ~/.zshenv file:

export PYTHONSTARTUP=$HOME/.startup.py

If you use bash, you’ll put that in your ~/.bashrc file. If you’re on Windows you’ll need to set an environment variable the Windows way.

With that variable set, I can now create a ~/.startup.py file that has Python code in it. That code will automatically run every time I launch a new Python REPL, whether within a virtual environment or outside of one.

My REPL keyboard shortcuts

The quick summary of my current modifications are:

Pressing Home moves to the first character in the code block
Pressing End moves to the last character in the code block
Pressing Alt+M moves to the first character on the current line
Pressing Shift+Tab removes common indentation from the code block
Pressing Alt+Up swaps the current line with the line above it
Pressing Alt+Down swaps the current line with the line below it
Pressing Ctrl+N inserts a specific list of numbers
Pressing Ctrl+F inserts a specific list of strings

If you’ve read my Python REPL shortcuts article, you know that we can use Ctrl+A to move to the beginning of the line and Ctrl+E to move to the end of the line. I already use those instead of the Home and End keys, so I decided to rebind Home and End to do something different.

The Alt+M key combination is essentially the same as Alt+M in Emacs or ^ in Vim. I usually prefer to move to the beginning of the non-whitespace in a line rather than to the beginning of the entire line.

The Shift+Tab functionality is basically a fancy wrapper around using textwrap.dedent: it dedents the current code block while keeping the cursor over the same character it was at before.

The Ctrl+N and Ctrl+F shortcuts make it easier for me to grab an example data structure to work with when teaching.

In addition to the above changes, I also modify my color scheme to work nicely with my Solarized Light color scheme in Vim.

I created a pyrepl-hacks library for this

My PYTHONSTARTUP file became so messy that I ended up creating a pyrepl-hacks library to help me with these modifications.

My PYTHONSTARTUP file now looks pretty much like this:

import pathlib as _pathlib, sys as _sys
_sys.path.append(str(_pathlib.Path.home() / ".pyhacks"))

try:
    import pyrepl_hacks as _repl
except ImportError:
    _repl = None  # We're on Python 3.12 or below
else:
    _repl.bind("Home", "home")
    _repl.bind("End", "end")
    _repl.bind("Alt+M", "move-to-indentation")
    _repl.bind("Shift+Tab", "dedent")
    _repl.bind("Alt+Down", "move-line-down")
    _repl.bind("Alt+Up", "move-line-up")
    _repl.bind_to_insert("Ctrl+N", "[2, 1, 3, 4, 7, 11, 18, 29]")
    _repl.bind_to_insert(
        "Ctrl+F",
        '["apples", "oranges", "bananas", "strawberries", "pears"]',
    )

    try:
        # Solarized Light theme to match vim
        _repl.update_theme(
            keyword="green",
            builtin="blue",
            comment="intense blue",
            string="cyan",
            number="cyan",
            definition="blue",
            soft_keyword="bold green",
            op="intense green",
            reset="reset, intense green",
        )
    except ImportError:
        pass  # We're on Python 3.13 or below

del _repl, _pathlib, _sys  # Avoid global REPL namespace pollution

That’s pretty short!

But wait… won’t this fail unless pyrepl-hacks is installed in every virtual environment and installed globally for every Python version on my machine?

That’s where that sys.path.append trick comes in handy…

Wait… let’s acknowledge the dragons 🐲

At this point I’d like to pause to note that all of this relies on using an implementation detail of Python that is deliberately undocumented because it is not designed to be used by end users.

The above code all relies on the _pyrepl module that was added in Python 3.13 (and optionally the _colorize module that was added in Python 3.14).

When I run a new future version of Python (for example Python 3.15) this solution may break. I’m willing to take that risk, as I know that I can always unset my shell’s PYTHONSTARTUP variable or clear out my startup file.

So, just be aware… here be (private undocumented implementation detail) dragons.

Monkey patching `sys.path` to allow importing `pyrepl_hacks`

I didn’t install pyrepl-hacks the usual way. Instead, I installed it in a very specific location.

I created a ~/.pyhacks directory and then installed pyrepl-hacks into that directory:

$ mkdir -p ~/.pyhacks
$ python -m pip install pyrepl-hacks --target ~/.pyhacks

In order for the pyrepl_hacks Python package to work, it needs to available within every Python REPL I might launch. Normally that would mean that it needs to be installed in every virtual environment that Python runs within. This trick avoids that constraint.

When Python tries to import a module, it iterates through the sys.path directory list. Any Python packages found within any of the sys.path directories may be imported.

So monkey patching sys.path within my PYTHONSTARTUP file allows pyrepl_hacks to be imported in every Python interpreter I launch:

from pathlib import Path
import sys
sys.path.append(str(Path.home() / ".pyhacks"))

With those 3 lines (or something like them) placed in my PYTHONSTARTUP file, all interactive Python interpreters I launch will be able to import modules that are located in my ~/.pyhacks directory.

Creating your own custom REPL commands

That’s pretty neat. But what if you want to invent your own REPL commands?

Well, the bind utility I’ve created in the pyrepl_hacks module can be used as a decorator for that.

This will make Ctrl+X followed by Ctrl+R insert import subprocess followed by subprocess.run("", shell=True) with the cursor positioned in between the double quotes after it’s all inserted:

import pyrepl_hacks as _repl

@_repl.bind(r"Ctrl+X Ctrl+R", with_event=True)
def subprocess_run(reader, event_name, event):
    """Ctrl+X followed by Ctrl+R will insert a subprocess.run command."""
    reader.insert("import subprocess\n")
    code = 'subprocess.run("", shell=True)'
    reader.insert(code)
    for _ in range(len(code) - code.index('""') - 1):
        _repl.commands.left(reader, event_name, event)

You can read more about the ins and outs of the pyrepl-hacks package in the readme file.

pyrepl-hacks is just a fancy wrapper

The pyrepl-hacks package is really just a fancy wrapper around Python’s _pyrepl and _colorize modules.

Why did I make a whole package and then modify my sys.path to use it, when I could have just used _pyrepl directly?

Three reasons:

To make creating new commands a bit easier (functions can be used instead of classes)
To make the key bindings look a bit nicer (I prefer "Ctrl+M" over r"\C-M")
To hide my hairy hacks behind a shiny API ✨

Before I made pyrepl-hacks, I implemented these commands directly within my PYTHONSTARTUP file by reaching into the internals of _pyrepl directly.

My PYTHONSTARTUP file before pyrepl-hacks was over 100 lines longer.

Try pyrepl-hacks and leave feedback

My hope is that the pyrepl-hacks library will be obsolete one day. Eventually the _pyrepl module might be renamed to pyrepl (or maybe just repl?) and it will have a well-documented friendly-ish public interface.

In the meantime, I plan to maintain pyrepl-hacks. As Python 3.15 is developed, I’ll make sure it continues to work. And I may add more useful commands if I think of any.

If you hack your own REPL, I’d love to hear what modifications you come up with. And if you have thoughts on how to improve pyrepl-hacks, please open an issue or get in touch.

Also, if you use Windows, please help me confirm whether certain keys work on Windows. Thanks!

Contributions and ideas welcome!

Customizing your Python REPL's color scheme (Python 3.14+)

2025-09-04T14:00:00-07:00

Did you know that Python 3.14 will include syntax highlighting in the REPL?

Python 3.14 is due to be officially released in about a month. I recommended tweaking your Python setup now so you’ll have your ideal color scheme on release day.

But… what if the default syntax colors don’t match the colors that your text editor uses?

Well, fortunately you can customize your color scheme!

Warning: I am recommending using an undocumented private module (it has an _-prefixed name) which may change in future Python versions. Do not use this module in production code.

Installing Python 3.14

Don’t have Python 3.14 installed yet?

If you have uv installed, you can run this command to launch Python 3.14:

$ uv run --python 3.14 python

That will automatically install 3.14 (if you don’t have it yet) and run it.

Setting a theme

I have my terminal colors set to the Solarized Light color palette and I have Vim use a Solarized Light color scheme as well.

The REPL doesn’t quite match my text editor by default:

The numbers, comments, strings, and keywords are all different colors than my text editor.

This code makes the Python REPL use nearly the same syntax highlighting as my text editor:

from _colorize import set_theme, default_theme, Syntax, ANSIColors

set_theme(default_theme.copy_with(
    syntax=Syntax(
        keyword=ANSIColors.GREEN,
        builtin=ANSIColors.BLUE,
        comment=ANSIColors.INTENSE_BLUE,
        string=ANSIColors.CYAN,
        number=ANSIColors.CYAN,
        definition=ANSIColors.BLUE,
        soft_keyword=ANSIColors.BOLD_GREEN,
        op=ANSIColors.INTENSE_GREEN,
        reset=ANSIColors.RESET + ANSIColors.INTENSE_GREEN,
    ),
))

Check it out:

Neat, right?

But… I want this to be enabled by default!

Using a `PYTHONSTARTUP` file

You can use a PYTHONSTARTUP file to run code every time a new Python process starts.

If Python sees a PYTHONSTARTUP environment variable when it starts up, it will open that file and evaluate the code within it.

I have this in my ~/.zshrc file to set the PYTHONSTARTUP environment variable to ~/.startup.py:

# Setup python-launcher to use startup file
export PYTHONSTARTUP=$HOME/.startup.py

In my ~/.startup.py file, I have this code:

def _main():
    """Everything's in a function to avoid polluting the global scope."""
    try:
        from _colorize import set_theme, default_theme, Syntax, ANSIColors
    except ImportError:
        pass  # Python 3.13 and below
    else:
        # Define Solarized Light colors
        solarized_light_theme = default_theme.copy_with(
            syntax=Syntax(
                keyword=ANSIColors.GREEN,
                builtin=ANSIColors.BLUE,
                comment=ANSIColors.INTENSE_BLUE,
                string=ANSIColors.CYAN,
                number=ANSIColors.CYAN,
                definition=ANSIColors.BLUE,
                soft_keyword=ANSIColors.BOLD_GREEN,
                op=ANSIColors.INTENSE_GREEN,
                reset=ANSIColors.RESET + ANSIColors.INTENSE_GREEN,
            ),
        )
        set_theme(solarized_light_theme)

_main()  # _main avoids name collision, in case python -i is used
del _main  # Delete _main from global scope

Note that:

I put all relevant code within a _main function so that the variables I set don’t remain in the global scope of the Python REPL (they will by default)
I call the _main function and then delete the function afterward, again so the _main variable doesn’t stay floating around in my REPL
I use try-except-else to ensure errors don’t occur on Python 3.13 and below

Also note that the syntax highlighting in the new REPL is not as fine-grained as many other syntax highlighting tools. I suspect that it may become a bit more granular over time, which may break the above code.

The _colorize module is currently an internal implementation detail and is deliberately undocumented. Its API may change at any time, so the above code may break in Python 3.15. If that happens, I’ll just update my PYTHONSTARTUP file at that point.

Packaging themes

I’ve stuck all of the above code in a ~/.startup.py file and I set the PYTHONSTARTUP environment variable on my system to point to this file.

Instead of manually updating a startup file, is there any way to make these themes installable?

Well, if a .pth file is included in Python’s site-packages directory, that file (which must be a single line) will be run whenever Python starts up. In theory, a package could use such a file to import a module and then call a function that would set the color scheme for the REPL. My dramatic package uses (cough abuses cough) .pth files in this way.

This sounds like a somewhat bad idea, but maybe not a horrible idea.

If you do this, let me know.

What’s your theme?

Have you played with setting a theme in your own Python REPL?

What theme are you using?

My PyCon US 2025 recap

2025-06-11T16:45:00-07:00

I’m pretty much fully back to normal life after PyCon US 2025.

I started writing this post shortly after PyCon, got side-tracked, and now I’m finally publishing it.

My very quick recap: I spent a ton of time at PyCon chatting with folks and I really enjoyed it. As Ned wrote, it’s like summer camp.

My pre-conference (Monday and Tuesday)

I flew in a couple days early (Monday May 12) because Breeze airlines had a non-stop flight from San Diego to Pittsburgh that only ran on Monday or Friday. I listened to The Upswing as I flew in and wondered how we can make our community less individualistic.

On Tuesday, I went on a walking tour of downtown Pittsburgh. Then I ate Indian food for lunch at Sree’s, which I chose because I saw they had great vegan lunch options and weren’t open for dinner. I was eating vegan during PyCon US (as I mostly do at home) and I remembered to do some research on vegan dinner restaurants in the area this year.

On Tuesday night, as CPython core devs started showing up in the Westin lobby, I went on a walk with Anthony Shaw. Anthony asked whether I was up for “an adventure” and I said yes without asking questions. We walked across the river to Randyland (a.k.a. “the mattress factory”). It was fun, interesting, and quite odd. We were the only ones there and had plenty of time to look around at all the interesting items and art pieces that Randy had collected.

Anthony and I did dinner at Condado’s and ran into more conference friends at the bar (Kattni, Rose, and Kojo).

My tutorial on decorators (Wednesday)

Wednesday was my tutorial day.

Attendees used sticky notes during the exercise sections to note when they needed help. I wrote about this system in my blog post of tips for giving a good PyCon tutorial.

The tutorial went well and the attendees seemed to enjoy it.

The newcomer’s orientation & education summit (Thursday)

Just before the opening reception on Thursday evening, Kojo, Sumana, and I ran the newcomer’s orientation. Figuring out what exactly we want to say to help orient folks to PyCon is always a bit challenging. A first-time PyCon attendee gave me some ideas for how we could do it even better next year. I’ve taken notes and will revisit them later.

I also attended a few talks during education summit earlier in the day. Reuven Lerner gave a talk on how he recommends his students use LLMs and I was pleased to hear many suggestions that are closely aligned with what I recommend as well as a few insights I hadn’t heard before.

My lightning talk (Friday)

I gave a lightning talk on Friday evening (the first talk day). I called it Systems, gates, and rage. It seemed to go over well. Folks occasionally told me throughout the conference that they enjoyed it. I won’t spoil the topic of my lightning talk (you’ll need to watch it) but it’s a topic that I’d been thinking about for a few months.

Rodrigo gave a meta lightning talk as the first talk of the first day. I gave a meta lightning talk last year in the same slot. I hope this becomes a tradition, where the first lightning talk is a talk about someone explaining how to give a lightning talk.

The keynotes

I really enjoyed the keynotes this year.

Cory Doctorow’s opening keynote was really thought-provoking and Lynn Root’s keynote was on the importance of “play”, which is a topic I’ve written about before to my mailing list.

The Marshall project keynote and The Carpentries keynote were also great, though I missed portions of each (beginning of one and end of another) and ended up watching the full videos online after the conference.

The hallway track

The “hallway track” is the way many PyCon attendees describe the discussions that happen organically in the hallway.

These discussions can happen at any time, including breaks, breakfast, lunch, and even during talks. I missed at least 3 talks this year because I was having a great discussion in the hallway and time got away from me.

PyCon’s venue in Pittsburgh included a great rooftop so the rooftop track / garden track was a lovely spin in the hallway track this year.

The open spaces

The open spaces are a very important part of PyCon for me.

Like the hallway track, the open spaces are mostly (usually) about having a conversation. Unlike the hallway track, the open spaces have a set topic.

I attended open spaces (and ran my own):

Education and Outreach Working Group Open Space: I’m part of this newly revived work group and we held this to hear topics that community members are interested in seeing the PSF focus on more deliberately
t-strings: Let’s build powerful templating together: I am interested in the new t-string feature in Python 3.14 and wanted to listen to this conversation (I contributed a little bit too)
Computer Assisted Reporting and Investigative journalism: I was curious to hear this conversation, which included journalists, government employees who respond to information requests, and many others
Python Skill-Building: I ran this open space on the roof during lunch time to chat about different Python skill-building services, including my own (Python Morsels)
Ask a Friendly Meat-Loving Vegan: I also ran this open space, inspired by a conversation I had the night before in a hotel lobby about animal farming, veganism, nutrition, and other related topics I’ve thought/read a lot about

I enjoyed meeting folks with similar interests and having fun and productive conversations (or at least listening to interesting conversations) in those open spaces.

Dinners, games, and hotel lobby conversations

I ate at Bae Bae’s, Rosewater Grill, and APTEKA during the conference. I also ate at Condado’s at least 3 nights.

APTEKA was a fun trek for vegan PierogiCon with mostly non-vegan folks. I’m sure the pierogis weren’t authentic, but we all loved the food.

After dinner every night I went to the Westin lobby and either played the Cabo card game or chatted with folks (1, 2, 3).

On missing talks

The keynotes and lightning talks are usually my favorite parts of PyCon. I tried to watch as many live keynotes and lightning talk sessions as I could this year. The morning lightning talks were hard to make it to because I kept sleeping in late enough that I missed most of them.

Ultimately, I watched very few live talks. I missed talks due to sleeping in, attending open spaces, visiting booths in the exhibit hall, getting lost in conversations in the hallway, and mid-day taking naps (in my hotel room, not during talks!).

Hynek has written about the downside of the hallway track and the importance of attending talks. I sympathize with Hynek’s argument that the hallway track is a trade off and there are downsides to missing talks, for both attendees and speakers. I am grateful that folks give talks and I want to support folks who do give talks, and yet, I often find myself attending few talks.

Ironically, the one talk I ended up taking a nap through (again, in my hotel room, not in the talk) was Hynek’s. My brain was fried by the time of his talk and I’m glad I was able to watch it online the week after PyCon. He apparently had a completely packed room as it was!

I did make sure to show up to a talk by a conference friend who I wanted to support (Michael Dahlberg’s talk on honeybee swarms). I have also binge-watched a dozen or so talks online that I had planned to attend but missed during PyCon.

You can watch all talks online now (at least I think they’re all up now?).

Attend a Python event

I’m looking forward to PyCon coming to Long Beach next year. I’m in San Diego and it will be nice to not fly to a PyCon for once!

I highly recommend attending local Python meetups, attending regional Python conferences, and/or attending PyCon US. Most in-person Python community events are pretty lovely. The Python community generally goes out of its way to be more welcoming than most tech events.

See you in Long Beach!

Which social network are we using for PyCon US this year?

2025-04-18T11:45:00-07:00

Last year I updated my having a great first PyCon post to note that Mastodon would likely be more popular than Twitter at PyCon.

My guess was correct. During PyCon US 2024, Mastodon overtook Twitter for the most posts on the #PyConUS hashtag.

In the fall of 2024, Bluesky really took off. It currently seems like Bluesky is now a bit more popular than Mastodon for Python posts in general… but that doesn’t necessarily mean it will be the most popular social media network during the conference.

This year I’m guessing that Mastodon will still be the most popular network for #PyConUS posts, though I wouldn’t be very surprised if Bluesky took the lead instead.

What do the social networks think?

I decided to pose this question to the various social networks via a poll. I made sure to share these polls using both the #Python and #PyConUS hashtags for visibility’s sake.

The results?

LinkedIn poll: LinkedIn isn’t sure whether Twitter is the new Twitter
Twitter poll Twitter (mostly) knows that Bluesky is the new Twitter
Bluesky poll Bluesky knows the really nerdy tech folks hang out on Mastodon
Mastodon poll Mastodon loves itself above all else

In other words, LinkedIn leans more toward Bluesky being the leader than Mastodon, Twitter leans toward Bluesky, but Bluesky and Mastodon both lean toward Mastodon being the leader during PyCon.

I didn’t mention Threads or other platforms because they didn’t seem like real contenders given where the most active PyCon-attending Python-posting folks seem to hang out in 2025.

The actual results

Here are the actual results of the polls.

Twitter

Bluesky

Mastodon

We’re still fragmented

Unfortunately, regardless of which network is the leader this year during PyCon US, we’re still going to be much more fragmented than we used to be. Twitter was the clear leader years ago and it very clearly isn’t anymore… at least not for PyCon US conference chatter.

I’ll be checking Mastodon and Bluesky and will post on at least Mastodon and possibly also Bluesky. I hope that other folks will also use one of these 2 social networks to organize dinners and gatherings! 🤞

Feel free to follow me on Mastodon and Bluesky during the conference.

Also let me know if you’d like to join my PyCon US starter pack on Bluesky. Lists on Mastodon require following and I prefer not to follow everyone I meet at PyCon so, unfortunately, I probably won’t have a Mastodon equivalent of this. 😢

I recommend checking the #PyConUS hashtag on both networks as well. Note that you can subscribe to hashtags on Mastodon which is pretty neat!

See you at PyCon!

If this will be your first PyCon US, I recommend signing up for both Mastodon and Bluesky and checking the #PyConUS hashtag during the conference.

Also, be sure to see my post on having a great first PyCon and see this additional tips.

Lazy self-installing Python scripts with uv

2024-12-09T11:15:10-08:00

I frequently find myself writing my own short command-line scripts in Python that help me with day-to-day tasks.

It’s so easy to throw together a single-file Python command-line script and throw it in my ~/bin directory!

Well… it’s easy, unless the script requires anything outside of the Python standard library.

Recently I’ve started using uv and my primary for use for it has been fixing Python’s “just manage the dependencies automatically” problem.

I’ll share how I’ve been using uv… first first let’s look at the problem.

A script without dependencies

If I have a Python script that I want to be easily usable from anywhere on my system, I typically follow these steps:

Add an appropriate shebang line above the first line in the file (e.g. #!/usr/bin/env python3)
Set an executable bit on the file (chmod a+x my_script.py)
Place the script in a directory that’s in my shell’s PATH variable (e.g. cp my_script.py ~/bin/my_script)

For example, here’s a script I use to print out 80 zeroes (or a specific number of zeroes) to check whether my terminal’s font size is large enough when I’m teaching:

#!/usr/bin/env python3
import sys

numbers = sys.argv[1:] or [80]
for n in numbers:
    print("0" * int(n))

This file lives at /home/trey/bin/0 so I can run the command 0 from my system prompt to see 80 0 characters printed in my terminal.

This works great! But this script doesn’t have any dependencies.

The problem: a script with dependencies

Here’s a Python script that normalizes the audio of a given video file and writes a new audio-normalized version of the video to a new file:

"""Normalize audio in input video file."""
from argparse import ArgumentParser
from pathlib import Path

from ffmpeg_normalize import FFmpegNormalize


def normalize_audio_for(video_path, audio_normalized_path):
    """Return audio-normalized video file saved in the given directory."""
    ffmpeg_normalize = FFmpegNormalize(audio_codec="aac", audio_bitrate="192k", target_level=-17)
    ffmpeg_normalize.add_media_file(str(video_path), audio_normalized_path)
    ffmpeg_normalize.run_normalization()


def main():
    parser = ArgumentParser()
    parser.add_argument("video_file", type=Path)
    parser.add_argument("output_file", type=Path)
    args = parser.parse_args()
    normalize_audio_for(args.video_file, args.output_file)


if __name__ == "__main__":
    main()

This script depends on the ffmpeg-normalize Python package and the ffmpeg utility. I already have ffmpeg installed, but I prefer not to globally install Python packages. I install all Python packages within virtual environments and I install global Python scripts using pipx.

At this point I could choose to either:

Create a virtual environment, install ffmpeg-normalize in it, and put a shebang line referencing that virtual environment’s Python binary at the top of my script file
Turn my script into a pip-installable Python package with a pyproject.toml that lists ffmpeg-normalize as a dependency and use pipx to install it

That first solution requires me to keep track of virtual environments that exist for specific scripts to work. That sounds painful.

The second solution involves making a Python package and then upgrading that Python package whenever I need to make a change to this script. That’s definitely going to be painful.

The solution: let uv handle it

A few months ago, my friend Jeff Triplett showed me that uv can work within a shebang line and can read a special comment at the top of a Python file that tells uv which Python version to run a script with and which dependencies it needs.

Here’s a shebang line that would work for the above script:

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "ffmpeg-normalize",
# ]
# ///

That tells uv that this script should be run on Python 3.12 and that it depends on the ffmpeg-normalize package.

Neat… but what does that do?

Well, the first time this script is run, uv will create a virtual environment for it, install ffmpeg-normalize into that venv, and then run the script:

$ normalize
Reading inline script metadata from `/home/trey/bin/normalize`
Installed 4 packages in 5ms
usage: normalize [-h] video_file output_file
normalize: error: the following arguments are required: video_file, output_file

Every time the script is run after that, uv finds and reuses the same virtual environment:

$ normalize
Reading inline script metadata from `/home/trey/bin/normalize`
usage: normalize [-h] video_file output_file
normalize: error: the following arguments are required: video_file, output_file

Each time uv runs the script, it quickly checks that all listed dependencies are properly installed with their correct versions.

Another script I use this for is caption, which uses whisper (via the Open AI API) to quickly caption my screencasts just after I record and edit them. The caption quality very rarely need more than a very minor edit or two (for my personal accent of English at least) even for technical like “dunder method” and via the API the captions generate very quickly.

See the inline script metadata page of the Python packaging users guide for more details on that format that uv is using (honestly I always just copy-paste an example myself).

uv everywhere?

I haven’t yet fully embraced uv everywhere.

I don’t manage my Python projects with uv, though I do use it to create new virtual environments (with --seed to ensure the pip command is available) as a virtualenvwrapper replacement, along with direnv.

I have also started using uv tool as a pipx replacement and I’ve considered replacing pyenv with uv.

uv instead of pipx

When I want to install a command-line tool that happens to be Python powered, I used to do this:

$ pipx countdown-cli

Now I do this instead:

$ uv tool install countdown-cli

Either way, I end up with a countdown script in my PATH that automatically uses its own separate virtual environment for its dependencies:

$ countdown --help
Usage: countdown [OPTIONS] DURATION

  Countdown from the given duration to 0.

  DURATION should be a number followed by m or s for minutes or seconds.

  Examples of DURATION:

  - 5m (5 minutes)
  - 45s (45 seconds)
  - 2m30s (2 minutes and 30 seconds)

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

uv instead of pyenv

For years, I’ve used pyenv to manage multiple versions of Python on my machine.

$ pyenv install 3.13.0

Now I could do this:

$ uv python install --preview 3.13.0

Or I could make a ~/.config/uv/uv.toml file containing this:

preview = true

And then run the same thing without the --preview flag:

$ uv python install 3.13.0

This puts a python3.13 binary in my ~/.local/bin directory, which is on my PATH.

Why “preview”? Well, without it uv doesn’t (yet) place python3.13 in my PATH by default, as this feature is currently in testing/development.

Update: uvrs for managing uv scripts

Since writing this post, I created uvrs, a tool for managing uv scripts.

With uvrs, you can use uvrs add and uvrs remove to manage your script’s inline dependencies, and uvrs init to create new scripts with the right metadata and executable permissions already set.

Most importantly, uvrs adds an exclude-newer timestamp to your script’s metadata, which limits dependency versions to those published before a given date. This means your scripts won’t unexpectedly break when a dependency releases a new version. When you’re ready to upgrade, uvrs stamp updates the timestamp and upgrades your dependencies.

uvrs also runs scripts with uv run --exact --script, ensuring the script’s environment always matches its metadata exactly.

If you’re using uv scripts regularly, uvrs makes them much easier to manage and more reproducible.

Self-installing Python scripts are the big win

I still prefer pyenv for its ability to install custom Python builds and I don’t have a preference between uv tool and pipx.

The biggest win that I’ve experienced from uv so far is the ability to run an executable script and have any necessary dependencies install automagically.

This doesn’t mean that I never make Python package out of my Python scripts anymore… but I do so much more rarely. I used to create a Python package out of a script as soon as it required third-party dependencies. Now my “do I really need to turn this into a proper package” bar is set much higher.

New Python Jumpstart course

2024-11-25T08:08:08-08:00

I’ve just recently launched a self-paced introduction to Python that is extremely hands-on. It’s called Python Jumpstart and it’s based on introductory Python curriculum that I have been iterating on for years.

Learn Python by writing Python code ✍

We do not learn by putting information into the brain. Our brains simply don’t retain knowledge that way.

Learning happens from repeatedly attempting to retrieve information from the brain. When it comes to Python, that means writing Python code.

So unlike most Python courses, Python Jumpstart is not focused on watching videos and rewriting code seen within videos. Instead, this new Python course is based around learning Python by writing Python code.

How Python Jumpstart is structured 🔬

Python Jumpstart includes learning Python by solving 46 short Python exercises.

Before each exercise, you’ll watch a 5 minute video explaining a new Python topic. You’ll then attempt the exercise to put one or more Python topics into practice.

You won’t solve many of the exercises on your first try and that’s okay. These exercises are designed to be revisited a few times over the course of weeks, until you’re satisfied with your solution.

This is not a sprint 📆

This structured path to Python proficiency, includes:

Carefully crafted exercises that build real understanding
Short, detailed explanations that won’t waste your time
A proven teaching approach that focuses on active learning
Spaced repetition to help concepts stick

This is not a course for impatient or passive learners. Learning takes repetitive effort spaced over many days and this course is structured to embrace that fact.

If you spend 30 minutes on Python Jumpstart each day, I estimate that you’ll complete this course in about 7 weeks. You’ll spend the large majority of that time writing Python code.

This course will take time, but it will be time well-spent.

Launch week special: 50% off until December 2 ⏰

Through Monday December 2, 2024, you can get lifetime access to Python Jumpstart for $99. After this launch week, Python Jumpstart will be $199.

Ready to jumpstart your Python learning journey?

Get Python Jumpstart for $99

Python Black Friday & Cyber Monday sales (2024)

2024-11-20T11:00:00-08:00

Ready for some Python skill-building sales?

This is my seventh annual compilation of Python learning deals.

Lots of Python sales

Here are Python-related sales that are live right now:

Python Jumpstart with Python Morsels: 50% off my brand new Python course, an introduction to Python that’s very hands-on ($99 instead of $199)
Talk Python: the annual everything bundle includes all courses for $240
Data School 40% off all Kevin’s courses or get a bundle with all 5 of his courses
Matt Harrison: get 25% off with GIVING25 plus every purchase will open a product for a scholarship recipient
Reuven Lerner: get 20% off Bamboo Weekly or 30% off any of his Weekly Python Exercise courses with code BF2024
Brian Okken: is offering 20% off his courses with code TURKEYSALE2024
Rodrigo 50% off Rodrigo’s all books bundle with code BF24
Mike Driscoll: 35% off Mike’s Python books and courses with code BF24
Test Driven: get the Flask course bundle for 30% off
The Python Coding Place: 40% off The Python Coding Book and 40% off a lifetime membership to The Python Coding Place with code black2024
Sundeep Agarwal: ~50% off Sundeep’s all book and Python bundles with code FestiveOffer
Benjamin Bennett Alexander: 60% off his Mastering Python Fundamentals book with code BLACK60
O’Reilly Media: 40% off the first year with code CYBERWEEK24 ($299 instead of $499)
Matplotlib Journey: a Python dataviz course that has a pre-launch promo right now (69€ instead of 149€)
Python Done Right: beta sale on this course ($99 instead of the eventual price of $299)
Pragmatic Bookshelf: 40% off sale with code turkeysale2024
No Starch: 35% off with code BLACKHATFRIDAY (Crash Course, Automate The Boring Stuff, etc.)
Manning is offering 50% off if you buy two items

Even more sales

Also see Adam Johnson’s Django-related Deals for Black Friday 2024 for sales on Adam’s books, courses from the folks at Test Driven, Django templates, and various other Django-related deals.

And for non-Python/Django Python deals, see the Awesome Black Friday / Cyber Monday deals GitHub repository and the BlackFridayDeals.dev website.

If you know of another sale (or a likely sale) please comment below or email me.

Adding keyboard shortcuts to the Python REPL

2024-10-28T07:15:00-07:00

I talked about the new Python 3.13 REPL a few months ago and after 3.13 was released. I think it’s awesome.

I’d like to share a secret feature within the Python 3.13 REPL which I’ve been finding useful recently: adding custom keyboard shortcuts.

This feature involves a PYTHONSTARTUP file, use of an unsupported Python module, and dynamically evaluating code.

In short, we may be getting ourselves into trouble. But the result is very neat!

Thanks to Łukasz Llanga for inspiring this post via his excellent EuroPython keynote talk.

The goal: keyboard shortcuts in the REPL

First, I’d like to explain the end result.

Let’s say I’m in the Python REPL on my machine and I’ve typed numbers =:

>>> numbers =

I can now hit Ctrl-N to enter a list of numbers I often use while teaching (Lucas numbers):

numbers = [2, 1, 3, 4, 7, 11, 18, 29]

That saved me some typing!

Getting a prototype working

First, let’s try out an example command.

Copy-paste this into your Python 3.13 REPL:

from _pyrepl.simple_interact import _get_reader
from _pyrepl.commands import Command

class Lucas(Command):

    def do(self):
        self.reader.insert("[2, 1, 3, 4, 7, 11, 18, 29]")

reader = _get_reader()
reader.commands["lucas"] = Lucas
reader.bind(r"\C-n", "lucas")

Now hit Ctrl-N.

If all worked as planned, you should see that list of numbers entered into the REPL.

Cool! Now let’s generalize this trick and make Python run our code whenever it starts.

But first… a disclaimer.

Here be dragons 🐉

Notice that _ prefix in the _pyrepl module that we’re importing from? That means this module is officially unsupported.

The _pyrepl module is an implementation detail and its implementation may change at any time in future Python versions.

In other words: _pyrepl is designed to be used by Python’s standard library modules and not anyone else. That means that we should assume this code will break in a future Python version.

Will that stop us from playing with this module for the fun of it?

It won’t.

Creating a `PYTHONSTARTUP` file

So we’ve made one custom key combination for ourselves. How can we setup this command automatically whenever the Python REPL starts?

We need a PYTHONSTARTUP file.

When Python launches, if it sees a PYTHONSTARTUP environment variable it will treat that environment variable as a Python file to run on startup.

I’ve made a /home/trey/.python_startup.py file and I’ve set this environment variable in my shell’s configuration file (~/.zshrc):

export PYTHONSTARTUP=$HOME/.python_startup.py

To start, we could put our single custom command in this file:

try:
    from _pyrepl.simple_interact import _get_reader
    from _pyrepl.commands import Command
except ImportError:
    pass  # Not in the new pyrepl OR _pyrepl implementation changed
else:
    class Lucas(Command):
        def do(self):
            self.reader.insert("[2, 1, 3, 4, 7, 11, 18, 29]")

    reader = _get_reader()
    reader.commands["lucas"] = Lucas
    reader.bind(r"\C-n", "lucas")

Note that I’ve stuck our code in a try-except block. Our code only runs if those _pyrepl imports succeed.

Note that this might still raise an exception when Python starts if the reader object’s command attribute or bind method change in a way that breaks our code.

Personally, I’d like to see those breaking changes occur print out a traceback the next time I upgrade Python. So I’m going to leave those last few lines without their own catch-all exception handler.

Generalizing the code

Here’s a PYTHONSTARTUP file with a more generalized solution:

try:
    from _pyrepl.simple_interact import _get_reader
    from _pyrepl.commands import Command
except ImportError:
    pass
else:
    # Hack the new Python 3.13 REPL!
    cmds = {
        r"\C-n": "[2, 1, 3, 4, 7, 11, 18, 29]",
        r"\C-f": '["apples", "oranges", "bananas", "strawberries", "pears"]',
    }
    from textwrap import dedent
    reader = _get_reader()
    for n, (key, text) in enumerate(cmds.items(), start=1):
        name = f"CustomCommand{n}"
        exec(dedent(f"""
            class _cmds:
                class {name}(Command):
                    def do(self):
                        self.reader.insert({text!r})
                reader.commands[{name!r}] = {name}
                reader.bind({key!r}, {name!r})
        """))
    # Clean up all the new variables
    del _get_reader, Command, dedent, reader, cmds, text, key, name, _cmds, n

This version uses a dictionary to map keyboard shortcuts to the text they should insert.

Note that we’re repeatedly building up a string of Command subclasses for each shortcut, using exec to execute the code for that custom Command subclass, and then binding the keyboard shortcut to that new command class.

At the end we then delete all the variables we’ve made so our REPL will start the clean global environment we normally expect it to have:

Python 3.13.0 (main, Oct  8 2024, 10:37:56) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> dir()
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

Is this messy?

Yes.

Is that a needless use of a dictionary that could have been a list of 2-item tuples instead?

Yes.

Does this work?

Yes.

Doing more interesting and risky stuff

Note that there are many keyboard shortcuts that may cause weird behaviors if you bind them.

For example, if you bind Ctrl-i, your binding may trigger every time you try to indent. And if you try to bind Ctrl-m, your binding may be ignored because this is equivalent to hitting the Enter key.

So be sure to test your REPL carefully after each new binding you try to invent.

If you want to do something more interesting, you could poke around in the _pyrepl package to see what existing code you can use/abuse.

For example, here’s a very hacky way of making a binding to Ctrl-x followed by Ctrl-r to make this import subprocess, type in a subprocess.run line, and move your cursor between the empty string within the run call:

    class _cmds:
        class Run(Command):
            def do(self):
                from _pyrepl.commands import backward_kill_word, left
                backward_kill_word(self.reader, self.event_name, self.event).do()
                self.reader.insert("import subprocess\n")
                code = 'subprocess.run("", shell=True)'
                self.reader.insert(code)
                for _ in range(len(code) - code.index('""') - 1):
                    left(self.reader, self.event_name, self.event).do()
    reader.commands["subprocess_run"] = _cmds.Run
    reader.bind(r"\C-x\C-r", "subprocess_run")

What keyboard shortcuts are available?

As you play with customizing keyboard shortcuts, you’ll likely notice that many key combinations result in strange and undesirable behavior when overridden.

For example, overriding Ctrl-J will also override the Enter key… at least it does in my terminal.

I’ll list the key combinations that seem unproblematic on my setup with Gnome Terminal in Ubuntu Linux.

Here are Control key shortcuts that seem to be complete unused in the Python REPL:

Ctrl-N
Ctrl-O
Ctrl-P
Ctrl-Q
Ctrl-S
Ctrl-V

Note that overriding Ctrl-H is often an alternative to the backspace key

Here are Alt/Meta key shortcuts that appear unused on my machine:

Alt-A
Alt-E
Alt-G
Alt-H
Alt-I
Alt-J
Alt-K
Alt-M
Alt-N
Alt-O
Alt-P
Alt-Q
Alt-S
Alt-V
Alt-W
Alt-X
Alt-Z

You can add an Alt shortcut by using \M (for “meta”). So r"\M-a" would capture Alt-A just as r"\C-a" would capture Ctrl-A.

Here are keyboard shortcuts that can be customized but you might want to consider whether the current default behavior is worth losing:

Alt-B: backward word (same as Ctrl-Left)
Alt-C: capitalize word (does nothing on my machine…)
Alt-D: kill word (delete to end of word)
Alt-F: forward word (same as Ctrl-Right)
Alt-L: downcase word (does nothing on my machine…)
Alt-U: upcase word (does nothing on my machine…)
Alt-Y: yank pop
Ctrl-A: beginning of line (like the Home key)
Ctrl-B: left (like the Left key)
Ctrl-E: end of line (like the End key)
Ctrl-F: right (like the Right key)
Ctrl-G: cancel
Ctrl-H: backspace (same as the Backspace key)
Ctrl-K: kill line (delete to end of line)
Ctrl-T: transpose characters
Ctrl-U: line discard (delete to beginning of line)
Ctrl-W: word discard (delete to beginning of word)
Ctrl-Y: yank
Alt-R: restore history (within history mode)

What fun have you found in `_pyrepl`?

Find something fun while playing with the _pyrepl package’s inner-workings?

I’d love to hear about it! Comment below to share what you found.

Django and the Python 3.13 REPL

2024-10-13T21:03:32-07:00

Your new Django project uses Python 3.13.

You’re really looking forward to using the new REPL… but python manage.py shell just shows the same old Python REPL. What gives?

Well, Django’s management shell uses Python’s code module to launch a custom REPL, but the code module doesn’t (yet) use the new Python REPL.

So you’re out of luck… or are you?

How stable do you need your `shell` command to be?

The new Python REPL’s code lives in a _pyrepl package. Surely there must be some way to launch the new REPL using that _pyrepl package!

First, note the _ before that package name. It’s _pyrepl, not pyrepl.

Any solution that relies on this module may break in future Python releases.

So… should we give up on looking for a solution, if we can’t get a “stable” one?

I don’t think so.

My shell command doesn’t usually need to be stable in more than one version of Python at a time. So I’m fine with a solution that attempts to use the new REPL and then falls back to the old REPL if it fails.

A working solution

So, let’s look at a working solution.

Stick this code in a management/commands/shell.py file within one of your Django apps:

"""Python 3.13 REPL support using the unsupported _pyrepl module."""
from django.core.management.commands.shell import Command as BaseShellCommand


class Command(BaseShellCommand):
    shells = ["ipython", "bpython", "pyrepl", "python"]

    def pyrepl(self, options):
        from _pyrepl.main import interactive_console
        interactive_console()

How it works

Django’s shell command has made it very simple to add support for your favorite REPL of choice.

The code for the shell command loops through the shells list and attempts to run a method with that name on its own class. If an ImportError is raised then it attempts the next command, stopping once no exception occurs.

Our new command will try to use IPython and bpython if they’re installed and then it will try the new Python 3.13 REPL followed by the old Python REPL.

If Python 3.14 breaks our import by moving the interactive_console function, then an ImportError will be raised, causing us to fall back to the old REPL after we upgrade to Python 3.14 one day. If instead, the interactive_console function’s usage changes (maybe it will require arguments) then our shell command will completely break and we’ll need to manually fix it when we upgrade to Python 3.14.

What’s so great about the new REPL?

If you’re already using IPython or BPython as your REPL and you’re enjoying them, I would stick with them.

Third-party libraries move faster than Python itself and they’re often more feature-rich. IPython has about 20 years worth of feature development and it has features that the built-in Python REPL will likely never have.

If you’re using the default Python REPL though, this new REPL is a huge upgrade. I’ve been using it as my default REPL since May and I love it. See my screencast on Python 3.13 for my favorite features in the new REPL.

P.S. for Python Morsels users: if you want to try using that code module, check out the (fairly advanced) replr or (even more advanced) replsync exercises.

Switching from virtualenvwrapper to direnv, Starship, and uv

2024-10-03T14:00:00-07:00

Earlier this week I considered whether I should finally switch away from virtualenvwrapper to using local .venv managed by direnv.

I’ve never seriously used direnv, but I’ve been hearing Jeff and Hynek talk about their use of direnv for a while.

After a few days, I’ve finally stumbled into a setup that works great for me. I’d like to note the basics of this setup as well as some fancy additions that are specific to my own use case.

My old virtualenvwrapper workflow

First, I’d like to note my old workflow that I’m trying to roughly recreate:

I type mkvenv3 <project_name> to create a new virtual environment for the current project directory and activate it
I type workon <project_name> when I want to workon that project: this activates the correct virtual environment and changes to the project directory

The initial setup I thought of allows me to:

Run echo layout python > .envrc && direnv allow to create a virtual environment for the current project and activate it
Change directories into the project directory to automatically activate the virtual environment

The more complex setup I eventually settled on allows me to:

Run venv <project_name> to create a virtual environment for the current project and activate it
Run workon <project_name> to change directories into the project (which automatically activates the virtual environment)

The initial setup

First, I installed direnv and added this to my ~/.zshrc file:

eval "$(direnv hook zsh)"

Then whenever I wanted to create a virtual environment for a new project I created a .envrc file in that directory, which looked like this:

layout python

Then I ran direnv allow to allow, as direnv instructed me to, to allow the new virtual environment to be automatically created and activated.

That’s pretty much it.

Unfortunately, I did not like this initial setup.

No shell prompt?

The first problem was that the virtual environment’s prompt didn’t show up in my shell prompt. This is due to a direnv not allowing modification of the PS1 shell prompt. That means I’d need to modify my shell configuration to show the correct virtual environment name myself.

So I added this to my ~/.zshrc file to show the virtual environment name at the beginning of my prompt:

# Add direnv-activated venv to prompt
show_virtual_env() {
  if [[ -n "$VIRTUAL_ENV_PROMPT" && -n "$DIRENV_DIR" ]]; then
    echo "($(basename $VIRTUAL_ENV_PROMPT)) "
  fi
}
PS1='$(show_virtual_env)'$PS1

Wrong virtual environment directory

The next problem was that the virtual environment was placed in .direnv/python3.12. I wanted each virtual environment to be in a .venv directory instead.

To do that, I made a .config/direnv/direnvrc file that customized the python layout:

layout_python() {
    if [[ -d ".venv" ]]; then
        VIRTUAL_ENV="$(pwd)/.venv"
    fi

    if [[ -z $VIRTUAL_ENV || ! -d $VIRTUAL_ENV ]]; then
        log_status "No virtual environment exists. Executing \`python -m venv .venv\`."
        python -m venv .venv
        VIRTUAL_ENV="$(pwd)/.venv"
    fi

    # Activate the virtual environment
    . $VIRTUAL_ENV/bin/activate
}

Loading, unloading, loading, unloading…

I also didn’t like the loading and unloading messages that showed up each time I changed directories. I removed those by clearing the DIRENV_LOG_FORMAT variable in my ~/.zshrc configuration:

export DIRENV_LOG_FORMAT=

The more advanced setup

I don’t like it when all my virtual environment prompts show up as .venv. I want ever prompt to be the name of the actual project… which is usually the directory name.

I also really wanted to be able to type venv to create a new virtual environment, activate it, and create the .envrc file for my automatically.

Additionally, I thought it would be really handy if I could type workon <project_name> to change directories to a specific project.

I made two aliases in my ~/.zshrc configuration for all of this:

venv() {
    local venv_name=${1:-$(basename "$PWD")}
    local projects_file="$HOME/.projects"

    # Check if .envrc already exists
    if [ -f .envrc ]; then
        echo "Error: .envrc already exists" >&2
        return 1
    fi

    # Create venv
    if ! python3 -m venv --prompt "$venv_name"; then
        echo "Error: Failed to create venv" >&2
        return 1
    fi

    # Create .envrc
    echo "layout python" > .envrc

    # Append project name and directory to projects file
    echo "${venv_name} = ${PWD}" >> $projects_file

    # Allow direnv to immediately activate the virtual environment
    direnv allow
}

workon() {
    local project_name="$1"
    local projects_file="$HOME/.projects"
    local project_dir

    # Check for projects config file
    if [[ ! -f "$projects_file" ]]; then
        echo "Error: $projects_file not found" >&2
        return 1
    fi

    # Get the project directory for the given project name
    project_dir=$(grep -E "^$project_name\s*=" "$projects_file" | sed 's/^[^=]*=\s*//')

    # Ensure a project directory was found
    if [[ -z "$project_dir" ]]; then
        echo "Error: Project '$project_name' not found in $projects_file" >&2
        return 1
    fi

    # Ensure the project directory exists
    if [[ ! -d "$project_dir" ]]; then
        echo "Error: Directory $project_dir does not exist" >&2
        return 1
    fi

    # Change directories
    cd "$project_dir"
}

Now I can type this to create a .venv virtual environment in my current directory, which has a prompt named after the current directory, activate it, and create a .envrc file which will automatically activate that virtual environment (thanks to that ~/.config/direnv/direnvrc file) whenever I change into that directory:

$ venv

If I wanted to customized the prompt name for the virtual environment, I could do this:

$ venv my_project

When I wanted to start working on that project later, I can either change into that directory or if I’m feeling lazy I can simply type:

$ workon my_project

That reads from my ~/.projects file to look up the project directory to switch to.

Switching to uv

I also decided to try using uv for all of this, since it’s faster at creating virtual environments. One benefit of uv is that it tries to select the correct Python version for the project, if it sees a version noted in a pyproject.toml file.

Another benefit of using uv, is that I should also be able to update the venv to use a specific version of Python with something like --python 3.12.

Here are the updated shell aliases for the ~/.zshrc for uv:

venv() {
    local venv_name
    local dir_name=$(basename "$PWD")

    # If there are no arguments or the last argument starts with a dash, use dir_name
    if [ $# -eq 0 ] || [[ "${!#}" == -* ]]; then
        venv_name="$dir_name"
    else
        venv_name="${!#}"
        set -- "${@:1:$#-1}"
    fi

    # Check if .envrc already exists
    if [ -f .envrc ]; then
        echo "Error: .envrc already exists" >&2
        return 1
    fi

    # Create venv using uv with all passed arguments
    if ! uv venv --seed --prompt "$@" "$venv_name"; then
        echo "Error: Failed to create venv" >&2
        return 1
    fi

    # Create .envrc
    echo "layout python" > .envrc

    # Append to ~/.projects
    echo "${venv_name} = ${PWD}" >> ~/.projects

    # Allow direnv to immediately activate the virtual environment
    direnv allow
}

Switching to starship

I also decided to try out using Starship to customize my shell this week.

I added this to my ~/.zshrc:

eval "$(starship init zsh)"

And removed this, which is no longer needed since Starship will be managing the shell for me:

# Add direnv-activated venv to prompt
show_virtual_env() {
  if [[ -n "$VIRTUAL_ENV_PROMPT" && -n "$DIRENV_DIR" ]]; then
    echo "($(basename $VIRTUAL_ENV_PROMPT)) "
  fi
}
PS1='$(show_virtual_env)'$PS1

I also switched my python layout for direnv to just set the $VIRTUAL_ENV variable and add the $VIRTUAL_ENV/bin directory to my PATH, since the $VIRTUAL_ENV_PROMPT variable isn’t needed for Starship to pick up the prompt:

layout_python() {
    VIRTUAL_ENV="$(pwd)/.venv"
    PATH_add "$VIRTUAL_ENV/bin"
    export VIRTUAL_ENV
}

I also made a very boring Starship configuration in ~/.config/starship.toml:

format = """
$python\
$directory\
$git_branch\
$git_state\
$character"""

add_newline = false

[python]
format = '([(\($virtualenv\) )]($style))'
style = "bright-black"

[directory]
style = "bright-blue"

[character]
success_symbol = "[\\$](black)"
error_symbol = "[\\$](bright-red)"
vimcmd_symbol = "[❮](green)"

[git_branch]
format = "[$symbol$branch]($style) "
style = "bright-purple"

[git_state]
format = '\([$state( $progress_current/$progress_total)]($style)\) '
style = "purple"

[cmd_duration.disabled]

I setup such a boring configuration because when I’m teaching, I don’t want my students to be confused or distracted by a prompt that has considerably more information in it than their default prompt may have.

The biggest downside of switching to Starship has been my own earworm-oriented brain. As I update my Starship configuration files, I’ve repeatedly heard David Bowie singing “I’m a Starmaaan”. 🎶

Ground control to major TOML

After all of that, I realized that I could additionally use different Starship configurations for different directories by putting a STARSHIP_CONFIG variable in specific layouts. After that realization, I made my configuration even more vanilla and made some alternative configurations in my ~/.config/direnv/direnvrc file:

layout_python() {
    VIRTUAL_ENV="$(pwd)/.venv"

    PATH_add "$VIRTUAL_ENV/bin"
    export VIRTUAL_ENV

    export STARSHIP_CONFIG=/home/trey/.config/starship/python.toml
}

layout_git() {
    export STARSHIP_CONFIG=/home/trey/.config/starship/git.toml
}

Those other two configuration files are fancier, as I have no concern about them distracting my students since I’ll never be within those directories while teaching.

You can find those files in my dotfiles repository.

The necessary tools

So I replaced virtualenvwrapper with direnv, uv, and Starship. Though direnv is doing most of the important work here. The use of uv and Starship were just bonuses.

I am also hoping to eventually replace my pipx use with uv and once uv supports adding python3.x commands to my PATH, I may replace my use of pyenv with uv as well.

Thanks to all who participated in my Mastodon thread as I fumbled through discovering this setup.

10-Week Hands-On Python Course

2024-08-20T14:20:00-07:00

Ever wished you could take an Intro to Python training with me, but you don’t work for a company with a generous training budget? I’m running a Python-learning program just for this situation.

Python High Five is a 10-week Python jumpstart program that starts this September.

Set aside the time to learn ⌚

One of the biggest problems for folks starting to learn Python is setting aside the time. And even if you do manage to set aside the time, you’ll often hit a roadblock where you feel confused.

Python High Five is a way to keep a daily learning habit and to get help when find yourself stuck.

This program is based around daily practice. Monday through Friday you’ll pick 30 minutes from your schedule, at any time that works you. During those 30 minutes, you’ll watch a 5 minute video, work on the day’s exercise, and reflect on your progress.

The most effective learning is hands-on 🖐️

Python High Five is all about learning through writing Python code. Each week we’ll dive deeper into Python, building upon what we’ve learned so far.

When you find yourself stuck you can get help through an asynchronous group chat and weekly office hour sessions. In addition to our weekly office hours together, I’ll check the chat each day, respond to questions, and provide guidance.

Proven learning techniques behind the scenes 📝

The daily check-ins allow for daily accountability. The group chat also provides both a community of peers to rely on, and guidance from an experienced Python trainer (me).

We’ll also be using proven learning techniques behind the scenes:

Retrieval practice: you don’t learn by putting information into your head, but by trying to take it out; for Python learning, that means writing code.
Spaced repetition: cramming is less effective than learning spaced out over time, which is why we’ll spend 30 minutes each weekday instead of spending a few hours every week.
Interleaving: each day’s exercise isn’t predictably themed because a bit of unpredictability can be really improve learning outcomes.
Elaboration: your daily check-in isn’t just about reflection: it’s also a helpful learning tool!

Plus, we’ll be working through curriculum I’ve been developing and iterating on for many years. I have taught these topics in many different settings to folks from many different backgrounds.

Form a daily learning habit 🔁

Any 10-week program will be just the start of a Python learning habit. You’ll need to keep up your Python after Python High Five ends, either by promptly applying your skills to a new project or diving deeper into Python with continued daily practice.

That’s why I’m offering an 80% discount for High Five attendees on one year of Python Morsels, which is my skill-building service designed to help deepen your Python skills every week. You can see more details on that here.

Ready to start your Python journey? ⛰️

Are you ready to start your Python journey with a solid foundation?

Read more about Python High Five and decide whether this is for you.

Keep in mind that while the program begins on September 9, enrollment closes on August 31. So check the FAQs and if you have additional questions, be sure to email me soon!

Quickly find the right datetime format code for your date

2024-08-05T11:30:00-07:00

I often find myself with a string representing a date and time and the need to create a format string that will parse this string into a datetime object.

I decided to make a tool that solves this problem for me: https://pym.dev/strptime

Finding the code to parse a date format with `strptime`

Here’s how I’m now using this new tool.

I find a date string in a random spreadsheet or log file that I need to parse. For example, the string 30-Jun-2024 20:09, which I recently found in a spreadsheet.

I then paste the string into the tool and watch the format appear:

Then I click on the date format to copy-paste it. That’s it!

This tool works by cycling through a number of common formats. It also works for dates without a time, like Jul 1, 2024.

This input field works great when you’re in need of a code for the datetime class’s strptime method (which parses dates). But what if you need a code for strftime (for formatting dates)?

Finding the code to format a date with `strftime`

If you don’t have a date but instead want to construct a date in a specific common format, scroll down the page a bit.

This page includes a table of common formats.

Click on the format to copy it. That’s it.

Playing with format codes

What if you have a date format already but you’re not sure what it represents?

Paste it in the box!

For example if you’re wondering what the %B in %B %d, %Y means, paste it in to see what that represent with the current date and time:

Other features

There are a few other hidden features in this tool:

After a date or date format is pasted, if it corresponds to one of the formats listed in the table of common formats, that row will be highlighted
Hitting the Enter key anywhere on the page will select the input field
Clicking on a date within the format table will fill that date into the input box
The bottom of the page includes links to other useful datetime formatting/parsing tools as well as a link to the relevant Python documentation

Thoughts? Feature requests?

What do you think of this tool?

Is this something you’d bookmark and use often? Is this missing a key feature that you would need for it to be valuable for your use?

Are there date and time formats you’d like to see that don’t seem to be supported yet?

Comment or email me to let me know!

Why does "python -m json" not work? Why is it "json.tool"?

2024-08-01T14:00:00-07:00

Update: upon the encouragement of a few CPython core team members, I opened a pull request to add this to Python 3.14.

Have you ever used Python’s json.tool command-line interface?

You can run python -m json.tool against a JSON file and Python will print a nicely formatted version of the file.

python -m json.tool example.json
[
    1,
    2,
    3,
    4
]

Why do we need to run json.tool instead of json?

The history of `python -m`

Python 3.5 added the ability to run a module as a command-line script using the -m argument (see PEP 338) which was implemented in Python 2.5. While that feature was being an additional feature/bug was accidentally added, alowing packages to be run with the -m flag as well. When a package was run with the -m flag, the package’s __init__.py file would be run, with the __name__ variable set to __main__.

This accidental feature/bug was removed in Python 2.6.

It wasn’t until Python 2.7 that the ability to run python -m package was re-added (see below).

The history of the `json` module

The json module was added in Python 2.6. It was based on the third-party simplejson library.

That library originally relied on the fact that Python packages could be run with the -m flag to run the package’s __init__.py file with __name__ set to __main__ (see this code from version 1.8.1).

When simplejson was added to Python as the json module in Python 2.6, this bug/feature could no longer be relied upon as it was fixed in Python 2.6. To continue allowing for a nice command-line interface, it was decided that running a tool submodule would be the way to add a command-line interface to the json package (discussion here).

Python 2.7 added the ability to run any package as a command-line tool. The package would simply need a __main__.py file within it, which Python would run as the entry point to the package.

At this point, json.tool had already been added in Python 2.6 and no attempt was made (as far as I can tell) to allow python -m json to work instead. Once you’ve added a feature, removing or changing it can be painful.

Could we make `python -m json` work today?

We could. We would just need to rename tool.py to __main__.py. To allow json.tool to still work also, would could make a new tool.py module that simply imports json.__main__.

We could even go so far as to deprecate json.tool if we wanted to.

Should we do this though? 🤔

Should we make `python -m json` work?

I don’t know about you, but I would rather type python -m json than python -m json.tool. It’s more memorable and easier to guess, if you’re not someone who has memorized all the command-line tools hiding in Python.

But would this actually be used? I mean, don’t people just use the jq tool instead?

Well, a sqlite3 shell was added to Python 3.12 despite the fact that third-party interactive sqlite prompts are fairly common.

It is pretty handy to have a access to a tool within an unfamiliar environment where installing yet-another-tool might pose a problem. Think Docker, a locked-down Windows machine, or any computer without network or with network restrictions.

Personally, I’d like to see python3 -m json work. I can’t think of any big downsides. Can you?

Update: pull request opened.

Too long; didn’t read

The “too long didn’t read version”:

Python 2.5 added support for the -m argument for modules, but not packages
A third-party simplejson app existed with a nice CLI that relied on a -m implementation bug allowing packages to be run using -m
Python 2.6 fixed that implementation quirk and broke the previous ability to run python -m simplejson
Python 2.6 also added the json module, which was based on this third-party simplejson package
Since python -m json couldn’t work anymore, the ability to run python -m json.tool was added
Python 2.7 added official support for python -m some_package by running a __main__ submodule
The json.tool module already existed in Python 2.6 and the ability to run python -m json was (as far as I can tell) never seriously considered

All thanks to git history and issue trackers

I discovered this by noting the first commit that added the tool submodule to simplejson, which notes the fact that this was for consistency with the new json standard library module.

Thank you git history. And thank you to the folks who brought us the simplejson library, the json module, and the ability to use -m on both a module and a package!

Also, thank you to Alyssa Coghlan, Hugo van Kemenade, and Adam Turner for reviewing my pull request to add this feature to Python 3.14. 💖

A beautiful Python monstrosity

2024-06-08T14:30:00-07:00

Creating performance tests for Python Morsels exercises is a frequent annoyance

I loathe writing automated tests for performance-related exercises because they’re always flaky. How flaky depends on the exercise, what I’m testing, and the time variability inherent in the particular Python features that a learner might use.

I came up with a solution for flaky tests recently, but it also makes my tests less readable. I then came up with a tool to improve the readability, but that has its own trade-offs.

The code I eventually came up with is a beautiful Python monstrosity.

    @attempt_n_times(10)
    def _():
        nonlocal micro_time, tiny_time
        micro_time = time(micro_numbers)
        tiny_time = time(tiny_numbers)
        self.assertLess(tiny_time, micro_time*n)

I’ll explain what that code does, but first let’s talk about why it’s needed.

The flaky performance tests

My flaky performance tests initially looked like this:

def test_some_test(self):
    n, m = 2.45, 2.04

    micro_time = time(micro_numbers)
    tiny_time = time(tiny_numbers)
    self.assertLess(tiny_time, micro_time*n)

    small_time = time(small_numbers)
    self.assertLess(small_time, tiny_time*n)
    self.assertLess(small_time, micro_time*n*m)

    medium_time = time(medium_numbers)
    self.assertLess(medium_time, micro_time*n*m*m)
    self.assertLess(medium_time, tiny_time*n*m)
    self.assertLess(medium_time, small_time*n)

The first block runs a performance test for the user’s function on a very small list and on a slightly larger list and then asserting that the slightly larger list didn’t take too much longer to run. The next two blocks run the same code on even larger lists and make further assertions about the relative times that the code took to run.

This roughly approximates the time complexity of this code.

Running performance checks in a loop

These performance checks need to:

Predictably fail for inefficient solutions
Predictably pass for efficient solutions
Run fast (within just a few seconds) even when the code is inefficient
Avoid the use of threading because they’ll be running on WebAssembly in the browser
Run consistently on pretty much any computer

These 5 requirements together have caused me countless headaches. I get the tests passing well, but they don’t always fail when they should. I get the tests failing and passing when they should, but then they’re too slow. And so on…

Notice the n and m factors in the above assertions:

    self.assertLess(small_time, micro_time*n*m)

If n and m are too big, we’ll get false positives (tests passing when they should fail). If n and m are too small, we’ll get false negatives (tests failing when they should pass).

To avoid both Type I and Type II errors, I decided to keep n and m small but attempt the assertion block multiple times.

Here’s the (far less flaky) revised code:

def test_some_test(self):
    n, m = 2.45, 2.04

    for attempts_left in reversed(range(10)):
        try:
            micro_time = time(micro_numbers)
            tiny_time = time(tiny_numbers)
            self.assertLess(tiny_time, micro_time*n)
            break
        except AssertionError:
            if attempts_left == 0:
                raise

    for attempts_left in reversed(range(5)):
        try:
            small_time = time(small_numbers)
            self.assertLess(small_time, tiny_time*n)
            self.assertLess(small_time, micro_time*n*m)
            break
        except AssertionError:
            if attempts_left == 0:
                raise

for attempts_left in reversed(range(3)):
    try:
        medium_time = time(medium_numbers)
        self.assertLess(medium_time, micro_time*n*m*m)
        self.assertLess(medium_time, tiny_time*n*m)
        self.assertLess(medium_time, small_time*n)
        break
    except AssertionError:
        if attempts_left == 0:
            raise

The for loop runs the code multiple times, the break statement stops the code as soon as the assertions all pass, and the except and if ensure that any assertion errors are suppressed until/unless we’re on the final iteration of the loop.

Let’s call this a for-try-break-except-if-raise pattern. It’s an absurdly verbose name fitting of absurdly verbose code.

This for-try-break-except-if-raise pattern works pretty well! But it’s not pretty.

Like many programmers, I believe that Don’t Repeat Yourself (DRY) need not apply to tests. Tests are allowed to be repetitive if the verbosity improves readability.

But there is so much noise in that code! I decided that removing some noise might improve readability. So I devised a helper utility to reduce the repetition.

In search of a solution

While pondering the repetitive noise in this code, I wondered what Python features I could use to abstract away this for-try-break-except-if-raise pattern.

Could I make a context manager and use a with block? That might help with the try-except, but context managers can’t run their code block multiple times, so that wouldn’t help with the for and the break. So a context manager is out.

Could I abstract this away into a looping helper by implementing a generator function? We are looping and generator functions can break early. But, a generator function can’t catch an exception that’s raised within the body of a loop. So a generator function wouldn’t work either.

What about a decorator? 🤔

Context managers and decorators both sandwich a block of code. But decorators sandwich functions and they have the power to run the same function repeatedly. A decorator might work!

Here’s a decorator that will run a given function up to 10 times (until no AssertionError is raised):

def try_10_times(function):
    def wrapper():
        for attempts_left in reversed(range(10)):
            try:
                return function()
            except AssertionError:
                if attempts_left == 0:
                    raise
    return wrapper

To use this decorator, we would need to define a function and then call that function:

@try_10_times
def assertions():
    micro_time = time(micro_numbers)
    tiny_time = time(tiny_numbers)
    self.assertLess(tiny_time, micro_time*n)

assertions()

This isn’t quite good enough though…

We need a pattern to run code N times (not necessarily exactly 10)
We reference the variables defined in each block in later blocks, so micro_time and tiny_time will need to be available outside that function
We need this function to run just one time right after it’s defined… could we do that automatically?

All 3 of these problems are solvable:

We need a decorator that accepts arguments
We need to use rarely seen nonlocal statement
We could have the decorator automatically call the decorated function

The final weird decorator

Here’s the decorator I ended up with:

def attempt_n_times(n):
    """
    Run tests multiple times if assertions are raised.

    Allows for more forgiving tests when assertions may be a bit flaky.
    """
    def decorator(function):
        """This looks like a decorator, but it actually runs the function!"""
        for attempts_left in reversed(range(n)):
            try:
                return function()
            except AssertionError:
                if attempts_left == 0:
                    raise
    return decorator

This decorator accepts an n argument which determines the maximum number of times the decorated function should be called. The decorator then calls the function repeatedly in a for loop and a try-except block. As soon as an AssertionError is not raised during one of these function calls, the looping stops.

The weirdest part about this decorator is that it calls the decorated function. Note that the decorator function doesn’t define a wrapper function within itself… it just runs code right away!

The resulting beautiful Python monstrosity

Here’s the final refactored test code:

def test_some_test(self):
    n, m = 2.45, 2.04
    micro_time = tiny_time = small_time = medium_time = 0

    @attempt_n_times(10)
    def _():
        nonlocal micro_time, tiny_time
        micro_time = time(micro_numbers)
        tiny_time = time(tiny_numbers)
        self.assertLess(tiny_time, micro_time*n)

    @attempt_n_times(5)
    def _():
        nonlocal small_time
        small_time = time(small_numbers)
        self.assertLess(small_time, tiny_time*n)
        self.assertLess(small_time, micro_time*n*m)

    @attempt_n_times(3)
    def _():
        nonlocal medium_time
        medium_time = time(medium_numbers)
        self.assertLess(medium_time, micro_time*n*m*m)
        self.assertLess(medium_time, tiny_time*n*m)
        self.assertLess(medium_time, small_time*n)

The attempt_n_times decorator immediately calls the function it decorates. Each function is defined and immediately called one or more times, in a try-except block within a loop.

That’s why we’ve named these functions with the throwaway _ name: we don’t care about the name of a function we’re never going to refer to again.

Also note the use of the nonlocal statement. Each function in Python has its own scope and all assignments assign to the local scope by default. That nonlocal variable pulls those variables to the scope of the outer function instead.

Compare the above code to the code just before this refactor:

def test_some_test(self):
    n, m = 2.45, 2.04

    for attempts_left in reversed(range(10)):
        try:
            micro_time = time(micro_numbers)
            tiny_time = time(tiny_numbers)
            self.assertLess(tiny_time, micro_time*n)
            break
        except AssertionError:
            if attempts_left == 0:
                raise

    for attempts_left in reversed(range(5)):
        try:
            small_time = time(small_numbers)
            self.assertLess(small_time, tiny_time*n)
            self.assertLess(small_time, micro_time*n*m)
            break
        except AssertionError:
            if attempts_left == 0:
                raise

    for attempts_left in reversed(range(3)):
        try:
            medium_time = time(medium_numbers)
            self.assertLess(medium_time, micro_time*n*m*m)
            self.assertLess(medium_time, tiny_time*n*m)
            self.assertLess(medium_time, small_time*n)
            break
        except AssertionError:
            if attempts_left == 0:
                raise

I find the refactored version easier to skim.

But that attempt_n_times decorator does abuse the decorator syntax. Decorators aren’t meant to call the function they’re decorating.

Is this misuse of decorators worth it?

Is this worth it?

Decorators aren’t supposed to immediately call the function they decorate. But there’s nothing stopping them from doing so. I feel that I’ve traded “normal code” for a beautiful monstrosity that’s easier to skim at a glance.

The attempt_n_times decorator is pretending that it’s a block-level tool by using a function because there’s no other way to invent such a tool in Python.

I think abstracting away the for-try-break-except-if-raise pattern was worth it, even though I ended up abusing Python’s decorator syntax in the process.

What do you think? Was that attempt_n_times abstraction worth it?

PyCon 2024 Reflection

2024-05-28T13:00:00-07:00

I traveled back home from PyCon US 2024 last week. This is my reflection on my time at PyCon.

Attempting to eat vegan

Since 2020, I’ve been gradually eating more plant-based and a few months ago I decided to take PyCon as an opportunity to attempt exclusively vegan eating outside my own home. As I noted on Mastodon, it was a challenge and I failed every day at least once but I found the experience worthwhile. Our food system is very dairy-oriented.

Staying hydrated and fed

One of the first things I did before heading to the convention center was walk to Target and buy snacks and drinks. When at PyCon, I prefer to spend 30 minutes and $20 to have a backup plan for last minute hydration and calories (even if not the greatest calories). I never quite know when I might sleep through breakfast, find lunch lacking, or wish I’d eaten more dinner.

A tutorial, an orientation, a lightning talk, and open spaces

My responsibilities at PyCon this year included teaching a tutorial and helping run the Newcomer’s Orientation with Kojo and Sumana.

Yngve and Marie offered to act as teaching assistants during my tutorial and I was very grateful for their help! Rodrigo and Krishna also offered to TA just before my tutorial started and I was extra grateful to have even more help than I’d expected. The attendees were mostly better prepared than I expected they would be, which was also great. It’s always great to spend less time on setup and more time exploring Python together.

The newcomer’s orientation the next day went well. We kept it fairly brief and were able to address about 10 minutes of audience questions before the opening reception started.

Once my PyCon responsibilities completed, I invented a few more (light) responsibilities for myself. 😅 I signed up to give a lightning talk on how to give a lightning talk. They slotted it as the first talk of the first lightning talk session on Friday night. I kept this talk pretty much the same as the one I presented DjangoCon 2016. I could have made the transitions fancier, but I decided to embrace the idea of simplicity with the hope that audience members might think “look if that first speaker can give such a simple and succinct presentation, maybe I can too.”

On Saturday I ran an open space on Python Learning. Some of you showed up because you’re on my mailing list or you’re paying Python Morsels subscribers. Many folks showed up because the topic was interesting, either as a learner or as a teacher. I really enjoyed the round-table-style conversation we had.

I also ran a Cabo Card game open space during lunch on Sunday on the 4th floor rooftop. Cabo is my usual conference ice breaker game and I played it at least a few nights in The Westin lobby as well.

Seeing conference friends, old and new

For me, PyCon is largely about having conversations. The talks and tutorials are great for starting me thinking about an idea. The hallway track, open spaces, and meals are great for continuing conversations about those ideas (or other ideas).

My first morning in Pittsburgh, I chatted with Naomi and Reuven. I’m glad I ran into them before the conference kicked off because (as often happens at PyCon) I only very briefly saw either of them during the rest of PyCon!

After my tutorial that afternoon, I did dinner with Marie, Yngve, and Rodrigo at Rosewater Mediterranean (good vegan options, assuming you enjoy falafel and various sauces). As sometimes happens at PyCon, another PyCon attendee, Sachin, joined our table because we noticed him eating on his own at a table near us and invited him to join us.

On Saturday, Melanie, David, Jay, and I had a sort of mini San Diego Python study group reunion dinner before inviting folks to join us for Cabo and Knucklebones one night. The 4 of us originally met each other (along with Carol and other wonderful Python folks) at the San Diego Python study group about 10 years ago.

I had some wonderful conversations about ways to improve the Python documentation over dinner (at Nicky’s Thai) on Sunday night with so many docs-concerned folks who I highly respect. I’m really excited that Python has the documentation editorial board and I’m hopeful that that board, with the help of many others community members, will usher in big improvements to the documentation in the coming years.

I also met a number of Internet acquaintances IRL for the first time at PyCon. I met Tereza and Jessica, who I know from our work in the PSF Code of Conduct workgroup. I met Steve Lott, who I originally knew as a prolific question-answerer. I also met Hugo, a CPython core dev, the Python 3.14 & 3.15 release manager, and a social media user (which is how I’ve primarily interacted with him because the Internet is occasionally lovely). I was also very excited to meet many Python Morsels members as well as folks who know me through my weekly Python tips newsletter.

I was grateful to chat with Hynek and Al about creating talks, YouTube videos, and other online content. I also enjoyed chatting with Glyph a bit about our experiences consulting and training and (in hindsight) wished I’d planned an open space for either consultants or trainers, both of which have been held at PyCon before but it just takes someone to stick it on the open space board.

Many folks I only saw very briefly (I said a quick hi and bye to Andrew over lunch during the sprints) and some I didn’t see at all (Frank was at PyCon but we never ran into each other). Some I essentially saw through playing a few rounds of Cabo (Thomas and Ethan among many others). We also ran into at least 4 other PyCon attendees in the airport on Tuesday afternoon, including Bob and Julian, who it’s always a pleasure to see.

A Mastodon-oriented PyCon

On Thursday night I had the feeling that the number of Mastodon posts I saw on the #PyConUS hashtag was greater than the number of Twitter posts. I (very unscientifically) counted up the number of posts I was seeing on each and found that my perception was correct: Mastodon seemed to slightly overtake Twitter at PyCon this year.

Over dinner on Wednesday, I tried to convince Marie, Yngve, and Rodrigo to get Mastodon accounts just to follow the hashtag during PyCon. I succeeded: Marie and Yngve and Rodrigo!

Mastodon will never be the social media platform. Its decentralized nature is too much of a barrier for many folks. However, it does seem to be used by enough somewhat nerdy Python folks to now be one the most used social media platform for PyCon posting.

The talks

I ended up spending little time in the talks during PyCon. This wasn’t on purpose. I just happened to attend many open spaces, take personal breaks, and end up in hallway conversations often. I did see many of the lightning talks live, as well as Jay, Simon, and Sumana’s keynotes (all of them were exceptional) and the opening and closing remarks. I also watched a few talks from my hotel room while taking breaks.

While I’m often a bit light on my talk load at PyCon, I do recommend folks attend a good handful of live talks during PyCon, as Jon and others recommend. I wish I had seen more talks live. I also wish I had attended a few open spaces that I missed.

At any one time, I know that I’m always missing about 90% of what’s scheduled during PyCon (if you include the talks and the open spaces). That’s assuming I don’t ditch the conference entirely for a few hours and walk across a bridge or ride a funicular (neither of which I did, as I stuck around the venue the whole time this year). I am glad I saw, did, and talked about everything I did, but there’s always something I wish I’d seen/done!

The sprints

Thanks to the documentation dinner, I had a couple documentation-related ideas in mind on the first day of sprints. But I’m also really excited about the new Python REPL coming in Python 3.13 (in case you can’t tell from how much I talk about it), so I sprinted on that instead. Łukasz assigned me the task of researching keyboard shortcuts that the new REPL is missing (compared to the current one on Linux and Mac) so I spent some time researching that. I got to see the REPL running on Anthony’s laptop on Windows and I am so excited that Windows support will be included before 3.13.0 lands! 🎉

Partly inspired by Carol Willing’s PyCon preview message, I also thanked Pablo, Łukasz, and Lysandros in-person for all their work on the new Python REPL. 🤗

Until next year

I’ll be keynoting at PyOhio this year.

Besides PyOhio, I’m not sure whether I’ll make it to another conference until PyCon US next year. I’d love to attend all of them, but I do have work and personal goals that need accomplishing too!

I hope to see you at PyCon US 2025! In the meantime, if you’re wishing we’d exchanged contact details or met in-person, please feel free to stay in touch through Mastodon, LinkedIn, my weekly emails, YouTube, or Twitter.

Trey Hunner - Python

Python Morsels now has spaced repetition

We learn by recalling, not by reading

Spaced repetition beats the forgetting curve

What Daily Recall does

You can use Daily Recall for free

Reorganizing Python's sys module

The overview: 9 new sys submodules

The sys package and its submodules

sys.cli

sys.imports

sys.io

sys.repl

sys.interpreter

sys.memory

sys.exceptions

sys.profile

sys.runtime

Could this actually be done?

The magic of module-level __setattr__

What would the transition look like?

A thought experiment, not a proposal

Using a ~/.pdbrc file to customize the Python Debugger

The aliases in my ~/.pdbrc file

A demo of these 5 aliases

~/.pdbrc isn’t as powerful as PYTHONSTARTUP

Python Black Friday & Cyber Monday sales (2025)

Python-related sales

Django-related sales

More on my sale: Python Morsels Lifetime Access

Python Morsels Lifetime Access Sale

A more structured way to improve your Python skills

How it works

What Python Morsels includes

Lock in lifetime access

Handy Python REPL Modifications

Creating a PYTHONSTARTUP file

My REPL keyboard shortcuts

I created a pyrepl-hacks library for this

Wait… let’s acknowledge the dragons 🐲

Monkey patching sys.path to allow importing pyrepl_hacks

Creating your own custom REPL commands

pyrepl-hacks is just a fancy wrapper

Try pyrepl-hacks and leave feedback

Customizing your Python REPL's color scheme (Python 3.14+)

Installing Python 3.14

Setting a theme

Using a PYTHONSTARTUP file

Packaging themes

What’s your theme?

My PyCon US 2025 recap

My pre-conference (Monday and Tuesday)

My tutorial on decorators (Wednesday)

The newcomer’s orientation & education summit (Thursday)

My lightning talk (Friday)

The keynotes

The hallway track

The open spaces

Dinners, games, and hotel lobby conversations

On missing talks

Attend a Python event

Which social network are we using for PyCon US this year?

What do the social networks think?

The actual results

LinkedIn

Twitter

Bluesky

Mastodon

We’re still fragmented

See you at PyCon!

Lazy self-installing Python scripts with uv

A script without dependencies

The problem: a script with dependencies

The solution: let uv handle it

uv everywhere?

uv instead of pipx

uv instead of pyenv

Update: uvrs for managing uv scripts

Self-installing Python scripts are the big win

New Python Jumpstart course

The overview: 9 new `sys` submodules

The `sys` package and its submodules

`sys.cli`

`sys.imports`

`sys.io`

`sys.repl`

`sys.interpreter`

`sys.memory`

`sys.exceptions`

`sys.profile`

`sys.runtime`

The magic of module-level `setattr`

The aliases in my `~/.pdbrc` file

`~/.pdbrc` isn’t as powerful as `PYTHONSTARTUP`

Monkey patching `sys.path` to allow importing `pyrepl_hacks`

Using a `PYTHONSTARTUP` file

Creating a `PYTHONSTARTUP` file

What fun have you found in `_pyrepl`?

How stable do you need your `shell` command to be?

Finding the code to parse a date format with `strptime`

Finding the code to format a date with `strftime`

The history of `python -m`

The history of the `json` module

Could we make `python -m json` work today?

Should we make `python -m json` work?