netrc does not work if $HOME is not set

[dmerejkowsky]

BPO	28334
Nosy	@terryjreedy, @vstinner, @berkerpeksag, @zhangyangyu, @dmerejkowsky
PRs	bpo-28334: fix netrc not working when $HOME is not set #123 bpo-28334: netrc() now uses expanduser() to find .netrc file #4537
Files	netrc-use-expanduser.patch netrc-use-expanduser-with-test.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-11-25.10:39:33.765>
created_at = <Date 2016-10-01.15:35:54.397>
labels = ['3.7', 'type-bug', 'library']
title = 'netrc does not work if $HOME is not set'
updated_at = <Date 2017-11-25.10:39:33.763>
user = 'https://github.com/dmerejkowsky'

bugs.python.org fields:

activity = <Date 2017-11-25.10:39:33.763>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2017-11-25.10:39:33.765>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2016-10-01.15:35:54.397>
creator = 'Dimitri Merejkowsky'
dependencies = []
files = ['44914', '44933']
hgrepos = []
issue_num = 28334
keywords = ['patch']
message_count = 9.0
messages = ['277824', '277847', '277894', '278276', '278438', '278447', '278932', '306950', '306951']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'vstinner', 'berker.peksag', 'xiang.zhang', 'Dimitri Merejkowsky']
pr_nums = ['123', '4537']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28334'
versions = ['Python 3.7']

[dmerejkowsky]

If $HOME is not set, netrc will raise an exception.

Attached patch fixes the problem by using os.path.expanduser instead

[berkerpeksag]

Thanks for the patch, Dimitri. I think this is a reasonable improvement. However, since we are changing the behavior of the netrc() class, I'm not sure this can be considered as a bug fix.

In any case, 3.3 and 3.4 are in security-fix-only mode so I'm going to remove them from the versions field.

We need two things from you to move this forward:

1. A test. It should go in Lib/test/test_netrc.py. You can use

       env = support.EnvironmentVarGuard()
       env.unset('HOME')

to test the new behavior.

2. A CLA form. You can sign it online at https://www.python.org/psf/contrib/contrib-form/

[dmerejkowsky]

However, since we are changing the behavior of the netrc() class, I'm not sure this can be considered as a bug fix.

I was not sure either. The patch does change behavior in subtle ways...

I've added a new patch, and sent the CLA form.

Thanks for your time!

[terryjreedy]

On Windows, HOME is both split into two variables and replaced by USERPROFILE.
C:\Users\Terry>set HOME
HOMEDRIVE=C:
HOMEPATH=\Users\Terry

C:\Users\Terry>set USERPROFILE
USERPROFILE=C:\Users\Terry

So if it make sense to run this on Windows*, I would call it a bug.

• I am guessing that Windows' FTP clients do not use .netrc.

The doc says "the file .netrc in the user’s home directory will be read." without qualification by "if $HOME is set". I don't remember how os.path.expanduser might otherwise find the home directory on unix and don't know what unix users might reasonbly expect.

[dmerejkowsky]

if it make sense to run this on Windows

I found this issue while running cross-platform code. I needed to store some credentials, did not mind having them in plain-text and I thought .netrc was a good place for this. (did not need to re-invent the wheel ...)

don't know what unix users might reasonbly expect.

Well, I guess as a bonus of this patch we could add a link to the os.path.expanduser section[1] in netrc documentation.

The behavior when $HOME is not set is properly documented there.

[1] https://docs.python.org/3/library/os.path.html#os.path.expanduser

[terryjreedy]

FWIW, some history: The $HOME lookup was in Guido's original 1998-12-22 patch:
if not file:
file = os.path.join(os.environ['HOME'], ".netrc")
The lookup was wrapped in try-except 4 years later to give a 'consistent error message'.

Module ntpath dates back to 1994. Expanduser only used HOMEDRIVE and HOMEPATH. Guido added support for HOME in 1997. Support for USERPROFILE was added a decade later.

Since the module is NOT marked 'Unix-only', I think it a bug to not use os.path.expanduser. Larry, Ned, do either of you have an opinion on whether the change should be made in current versions or only 3.7?

[dmerejkowsky]

During review SilentGhost suggested that maybe a test was not essential.

In any case, I think patching documentation about the new behavior won't hurt.

Do you want me to do that?

[berkerpeksag]

New changeset 8d9bb11 by Berker Peksag in branch 'master':
bpo-28334: netrc() now uses expanduser() to find .netrc file (GH-4537)
8d9bb11

[berkerpeksag]

Thanks, Dimitri.