Repository navigation
Stance (or discussion) on src/ directory #320
Description
Is there an official stance on the "src/" directory thing? I'm all for it (and @hynek seems to agree) but I haven't found any discussion or explanation about it from the PyPA. The guide doesn't mention it and the sample project doesn't use it. Even if the stance is "don't use it" (which I would disagree with) then I'd love to see this discussed in the official guide so we can refer to it when people need to make up their mind or argue it.
Reacted by Andriy Drozdyuk, James Myatt, Maximilian Roos, Dominik Maier, John Hagen and TW
Activity
ctheune commentedon Jun 7, 2017
Some references on the discussion that support the "pro" argument:
pfmoore commentedon Jun 7, 2017
I don't think there's a consensus on the best practice, and the packaging guide isn't really intended to cover general development practices, which this is. There's no discussion on testing or documentation, for similar reasons.
theacodes commentedon Jun 7, 2017
Based on the doc plan in #317, this is actually a good candidate for a discussion topic. I would be willing to accept a PR to add this topic.
pfmoore commentedon Jun 7, 2017
Nice, @jonparrott - I hadn't seen that doc plan. Agreed this would make a good discussion topic. Another one I've seen come up in the past is whether to put tests in the package or in a separate test directory.
theacodes commentedon Jun 7, 2017
@pfmoore for sure, I'd be happy to have that topic as well.
theacodes commentedon Jun 23, 2017
@ctheune if you or @hynek are interested in contributing this topic, I would love to review and merge it. :)
pfmoore commentedon Jun 23, 2017
I'd also be interested in reviewing this. I tried a layout using
srcjust recently and found there were some things that were fiddly to get right (coverage was one), so I have a test case I could use to see how the recommendations work in practice :-)hynek commentedon Jun 23, 2017
Have you seen https://hynek.me/articles/testing-packaging/ Paul? I hoped to de-fiddle the process there...
pfmoore commentedon Jun 23, 2017
@hynek Yes I have. The problem I had was not with combining coverage, but with the basic coverage output from a single tox environment. It's possible it was just lack of familiarity with running coverage under tox, but from what I recall the output had some rather badly nested directories, when I was after filenames relative to
src(somypkg/__init__.py). I backed out the implementation of coverage, so I don't have it to hand at the moment, but could reimplement it if needed.ctheune commentedon Jun 28, 2017
Thanks for picking up that topic - happy to contribute although I can't promise to be very responsive at times (busy company and small child make my FLOSS contributions a bit spotty at the moment).
I haven't run into the coverage issue before - I usually have both a tox and non-tox pytest setup and ensure my coverage is right in the basic setup. I see the need for checking coverage of Python--version-specific code paths, though.
What's the next step for contributing here?
theacodes commentedon Jun 28, 2017
pradyunsg commentedon Aug 25, 2017
Is anyone working on this right now?
theacodes commentedon Aug 25, 2017
170 remaining items
pradyunsg commentedon May 27, 2019
If anyone has any concerns with this, please let us know. It'll be useful to discuss things if we've missed something here, or something was left un-addressed.
asmeurer commentedon May 27, 2019
There were several dissenting opinions to PyPA unilaterally recommending src/ layout. I thought the agreement here was to at most document the advantages/disadvantages of each, but not to recommend one way or the other. Just because a "unanimous" decision was reached in an in-person of PyPA members, that supersedes the lengthy discussion from the community that happened here?
pradyunsg commentedon May 27, 2019
I think it was to at least document the trade-offs. ;)
Anyway, the action items from the meeting are to document the trade-offs and updating the guide to show that src/ is a valid, reasonable layout.
Many of the points made here were taken into account and discussed in person. There was no "decision" made at the summit, but the room did reach consensus on a position we felt is reasonable, which I've put up here for further discussion. We obviously can't have everyone who wants to say something, be at the summit, which is why binding decisions weren't made in the summit. :)
I'll note that there were more non-PyPA folks in the Packaging Mini-Summit than PyPA members.
asmeurer commentedon May 27, 2019
If the goal here is simply to document tradeoffs of both approaches as valid, that's sounds good to me. @certik started a PR for that, though I had thought it was decided to not merge it (but I admit the discussions got quite lengthy and I stopped following them at some point, so I could be wrong about that).
But that isn't what this sounds like:
(emphasis added)
That sounds like you want the PyPA to officially say src/ is correct and non-src is incorrect. Given the large number of popular projects that use non-src, plus the reasons I and others discussed earlier in this thread (n.b., if you're cmd-f'ing on this page, first search for "hidden items" to make GitHub show all the comments), I think it's rather contentious to do that.
pfmoore commentedon May 27, 2019
To be honest, there's never going to be a consensus. Anyone expecting one is frankly being naive - just look at the tone of the discussions here. Although I just scanned the full thread here, and I only see 2 people really strongly against the
srclayout (not that I see that many participants at all, most of the debate here is fuelled by a very few people).But the packaging guide (and PyPA as custodians of that guide) must give guidance on how to create a new project - it's fundamental - and by doing so it will endorse a particular layout. Ideally, if a consensus were possible, we'd describe that. But in the absence of a consensus, we still have to take a stance. After long debate (most of which happened here), a course of action has been agreed (at PyCon, certainly, but not by PyPA members alone[1]). It's being reported back here.
People are welcome to write their own guides, recommending whatever layout they prefer. No-one has to follow the layout described in the packaging guide. But the debates have gone on long enough IMO and no-one is changing their position, saying anything new, or offering a better compromise. So I think it's time to put this discussion to bed, and move on.
[1] Note, I wasn't present at that discussion, for the record.
pradyunsg commentedon May 27, 2019
Ugh. I literally just wanted to document what we was discussed and agreed upon at the summit so that it doesn't get lost. If you don't agree with that, cool. Don't push back against me personally, for documenting that.
If you (or anyone else really) want to continue suggesting that I'm somehow trying to silence/subvert parts of this discussion, I'm not going to bother responding.
@certik's PR adds a guide but we need a discussion topic. @theacodes elaborated on that here: #320 (comment)
pradyunsg commentedon May 27, 2019
+1
Thanks @pfmoore for writing that! This is indeed a great summary of the situation here.
theacodes commentedon May 27, 2019
I generally dislike doing this, but I'm going to lock this conversation.
Here's our next steps:
@pradyunsg is working on turning @certik's guide into a discussion. After that is merged, I'll accept guides for src and non-src layouts from any authors.
After that, only at the consensus and request of the packaging working group will I update the primary tutorials or sample application to reflect
srclayout. In the absence of consensus precedence takes priority. If this is something you feel strongly about, please start a discussion in that work group's channels.Thanks, all.
pradyunsg commentedon Sep 30, 2022
This had fallen pretty low on my TODO list, but I finally got to filing the PR today: #1150