test_dbm fails when run directly

[serhiy-storchaka]

BPO	31026
Nosy	@terryjreedy, @vstinner, @larryhastings, @serhiy-storchaka, @bmoyles, @miss-islington
PRs	bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. #6632 [3.7] bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632) #6634 [3.6] bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632) #6635

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-04-29.14:08:20.862>
created_at = <Date 2017-07-25.12:10:17.055>
labels = ['3.7', '3.8', 'type-bug', 'tests']
title = 'test_dbm fails when run directly'
updated_at = <Date 2019-10-29.08:00:50.785>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2019-10-29.08:00:50.785>
actor = 'larry'
assignee = 'none'
closed = True
closed_date = <Date 2018-04-29.14:08:20.862>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2017-07-25.12:10:17.055>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 31026
keywords = ['patch']
message_count = 13.0
messages = ['299055', '299058', '299061', '299073', '299442', '301740', '301756', '315888', '315892', '315896', '315897', '355591', '355629']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'vstinner', 'larry', 'serhiy.storchaka', 'bmoyles', 'miss-islington']
pr_nums = ['6632', '6634', '6635']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31026'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[serhiy-storchaka]

$ ./python Lib/test/test_dbm.py
..BDB0004 fop_read_meta: @test_29975_tmp_ndbm.db: unexpected file type or format
..EBDB3028 @test_29975_tmp.db: unable to flush: No such file or directory
..EBDB3028 @test_29975_tmp.db: unable to flush: No such file or directory
.E.........

======================================================================
ERROR: test_anydbm_creation (main.TestCase-dbm.ndbm)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 73, in test_anydbm_creation
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_modification (main.TestCase-dbm.ndbm)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 88, in test_anydbm_modification
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

======================================================================
ERROR: test_anydbm_read (main.TestCase-dbm.ndbm)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_dbm.py", line 94, in test_anydbm_read
    self.read_helper(f)
  File "Lib/test/test_dbm.py", line 114, in read_helper
    self.assertEqual(self._dict[key], f[key.encode("ascii")])
KeyError: b'0'

---

Ran 19 tests in 0.683s

FAILED (errors=3)

[vstinner]

On my Linux (Fedora 25), "./python Lib/test/test_dbm.py" and "./python -m test test_dbm" both succeed.

[vstinner]

Which dbm implementation are you using? Do you have the following modules?

haypo@selma$ ./python 
Python 3.7.0a0 (heads/master:3924377, Jul 25 2017, 14:17:41) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _gdbm
>>> import _dbm

[serhiy-storchaka]

>>> import _gdbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_gdbm'
>>> import _dbm
>>>

[terryjreedy]

OK on Windows, skipping test_whichdb_ndbm (neither _dbm nor _gdbm are present.)

[bmoyles]

I ran into the same problem when running the test suite against a fresh Python 3.6.2 build on Ubuntu 16.04. It specifically appears to be a problem with ndbm and retrieving an empty byte string value:

Python 3.6.2 (default, Sep  8 2017, 18:31:28)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbm.gnu, dbm.ndbm
>>> g = dbm.gnu.open('gnudb', 'c')
>>> g['0'.encode('ascii')] = b''
>>> g.keys()
[b'0']
>>> g['0'.encode('ascii')]
b''
>>>
>>> n = dbm.ndbm.open('ndbm', 'c')
>>> n['0'.encode('ascii')] = b''
>>> n.keys()
[b'0']
>>> n['0'.encode('ascii')]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: b'0'

Same behavior when using the distribution's stock Python 3.5.2 so I suspect there's a behavioral change in libdb that's not accounted for?

[bmoyles]

Actually, I was able to make things happy by removing libdb5.3-dev and solely relying on libgdbm3-dev (which suggests that Debian upstream may want to pare back their build requirements rather than disable the dbm test as they appear to do now).

[serhiy-storchaka]

The same error is now occurred in Ubuntu 18.04 by default.

The proposed patch moves the test for the empty value to separate methods and skips them if dbm.ndbm uses Berkeley DB.

[serhiy-storchaka]

New changeset 70af06c by Serhiy Storchaka in branch 'master':
bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
70af06c

[miss-islington]

New changeset e57d3e3 by Miss Islington (bot) in branch '3.7':
bpo-31026: Fix test_dbm if dbm.ndbm is build with Berkeley DB. (GH-6632)
e57d3e3