How should I structure a growing Python project?

It's not really related to Python at all. For any language, you will use an object-oriented design for the sole reason that it scales well with larger projects. If every part of the program is written with lots of OO discipline, then the size of it won't matter since every module just ought to take care of it's own designated task and not worry about the outside world. This means that maintenance turns localized and bugs don't as easily escalate all over the program.

Similarly, if not enough care about design was taken from the start, then all design problems will escalate, "tight coupling" between unrelated parts will increase etc etc. It is good practice to always write production quality code from day 1 and not starting by hacking together some quick & dirty "debug release" code with the intention to clean it up later. "Just coding away" is the root of all evil.

The process of re-designing bad code is often called "refactoring", which is a snobby way of saying "I didn't consider requirements and/or design enough when I first wrote this bad code". Those who claim that "refactoring" is some sort of natural, mandatory part of the development process are probably not very experienced. With experience you'll learn that spending some extra week on requirements and design will save far more time later down the road. Because rewriting everything at a later stage takes lots of time and nobody wants to pay you for doing the work twice either.

If the code was originally well-designed but "scope creep", extra features or quick & dirty patches challenge that design, then that's known as code rot and it happens when you continuously maintain something over time. Particularly when abandoning discipline or when rolling out changes under time pressure. You might have to rewrite parts of the program from scratch to get rid of that. Ideally you should not abandon discipline when making a new extra feature and you should probably test it much more carefully than you tested the original code. A significant part of all bugs out there happen during maintenance.

In addition to paying extra attention to requirements and design, which are by far the most important parts, you also need:

1. A coding style guide for how you should write code.
2. Coding rules for which language/library features to avoid or treat with care.
3. A documented procedure for how, when & where you do commits to the version control system.
4. A documented procedure for how you publish/deploy programs.
5. Peer code review and/or static/dynamic analyzers are great ways to reduce bugs.

posted 22 days ago

CC BY-SA 4.0

Lundin‭

9680 reputation 42 131 1121 181

Official

The official Python documentation has material that may be helpful:

• Tutorials:
  • Modules, including § Packages
  • Virtual Environments and Packages
• The Python Packaging User Guide, especially Packaging Python Projects

A consistent style is important. See PEP 8 – Style Guide for Python Code. It doesn't say much about structure, but § Public and Internal Interfaces is relevant. There are tools that can automatically format your code, such as autopep8 (which makes minimal changes) and Black (which may make sweeping changes).

Static typing is helpful, including e.g. for IDE integration. See Static Typing with Python and the typing module.

Other

It's important to understand design principles for scalability. For example, dependency injection means things that you build are flexible. (As a sidenote, I've always disliked that name. I wouldn't really call it "injection" when it's usually done by just passing. I like to think of it like "if you hire a carpenter to build a shed, the foundation should already be poured, instead of having the carpenter hire someone else to do it".) SOLID is a good core set of principles for OOP.

Finally, I hope you're already using a version-control system, but if not, try starting with Git (which the CPython project itself uses, and which the venv module integrates with).

posted 22 days ago

CC BY-SA 4.0

21d ago

wjandrea‭

41 reputation 0 2 4 11

Split code by responsibility, use packages and modules, separate logic, data, and interfaces, follow MVC or layered patterns, keep tests separate, and refactor early as the project grows.

posted 14 days ago

CC BY-SA 4.0

psychicganeshbaba‭

-7 reputation 0 1 -6 0