bpo-33729: Fix issues with arguments parsing in hashlib. (GH-8346)

* help(hashlib) didn't work because of incorrect module name in blake2b and
  blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
  accepted keyword argument "string" for binary data, but documented as
  accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
  keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
  which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
  positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
  shake_*() and keccak_*() incorrectly referred to sha3_224.

Also made the following enhancements:

* More accurately specified input and result types for strings, bytes and
  bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.

Author: serhiy-storchaka

Doc/library/hashlib.rst

@@ -101,7 +101,7 @@ More condensed:
 
 .. function:: new(name[, data])
 
-   Is a generic constructor that takes the string name of the desired
+   Is a generic constructor that takes the string *name* of the desired
    algorithm as its first parameter.  It also exists to allow access to the
    above listed hashes as well as any other algorithms that your OpenSSL
    library may offer.  The named constructors are much faster than :func:`new`
@@ -162,10 +162,10 @@ A hash object has the following attributes:
 A hash object has the following methods:
 
 
-.. method:: hash.update(arg)
+.. method:: hash.update(data)
 
-   Update the hash object with the object *arg*, which must be interpretable as
-   a buffer of bytes.  Repeated calls are equivalent to a single call with the
+   Update the hash object with the :term:`bytes-like object`.
+   Repeated calls are equivalent to a single call with the
    concatenation of all the arguments: ``m.update(a); m.update(b)`` is
    equivalent to ``m.update(a+b)``.
 
@@ -206,7 +206,7 @@ by the SHAKE algorithm.
 .. method:: shake.digest(length)
 
    Return the digest of the data passed to the :meth:`update` method so far.
-   This is a bytes object of size ``length`` which may contain bytes in
+   This is a bytes object of size *length* which may contain bytes in
    the whole range from 0 to 255.
 
 
@@ -262,9 +262,10 @@ include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_.
    The function provides scrypt password-based key derivation function as
    defined in :rfc:`7914`.
 
-   *password* and *salt* must be bytes-like objects. Applications and
-   libraries should limit *password* to a sensible length (e.g. 1024). *salt*
-   should be about 16 or more bytes from a proper source, e.g. :func:`os.urandom`.
+   *password* and *salt* must be :term:`bytes-like objects
+   <bytes-like object>`.  Applications and libraries should limit *password*
+   to a sensible length (e.g. 1024).  *salt* should be about 16 or more
+   bytes from a proper source, e.g. :func:`os.urandom`.
 
    *n* is the CPU/Memory cost factor, *r* the block size, *p* parallelization
    factor and *maxmem* limits memory (OpenSSL 1.1.0 defaults to 32 MiB).
@@ -305,20 +306,20 @@ Creating hash objects
 New hash objects are created by calling constructor functions:
 
 
-.. function:: blake2b(data=b'', digest_size=64, key=b'', salt=b'', \
+.. function:: blake2b(data=b'', *, digest_size=64, key=b'', salt=b'', \
                 person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0,  \
                 node_depth=0, inner_size=0, last_node=False)
 
-.. function:: blake2s(data=b'', digest_size=32, key=b'', salt=b'', \
+.. function:: blake2s(data=b'', *, digest_size=32, key=b'', salt=b'', \
                 person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0,  \
                 node_depth=0, inner_size=0, last_node=False)
 
 
 These functions return the corresponding hash objects for calculating
 BLAKE2b or BLAKE2s. They optionally take these general parameters:
 
-* *data*: initial chunk of data to hash, which must be interpretable as buffer
-  of bytes.
+* *data*: initial chunk of data to hash, which must be
+  :term:`bytes-like object`.  It can be passed only as positional argument.
 
 * *digest_size*: size of output digest in bytes.
 
@@ -427,7 +428,7 @@ object, and, finally, get the digest out of the object by calling
 
 
 As a shortcut, you can pass the first chunk of data to update directly to the
-constructor as the first argument (or as *data* keyword argument):
+constructor as the positional argument:
 
     >>> from hashlib import blake2b
     >>> blake2b(b'Hello world').hexdigest()

Lib/hashlib.py

@@ -25,18 +25,18 @@
 sha384 and sha512 will be slow on 32 bit platforms.
 
 Hash objects have these methods:
- - update(arg): Update the hash object with the bytes in arg. Repeated calls
-                are equivalent to a single call with the concatenation of all
-                the arguments.
- - digest():    Return the digest of the bytes passed to the update() method
-                so far.
- - hexdigest(): Like digest() except the digest is returned as a unicode
-                object of double length, containing only hexadecimal digits.
- - copy():      Return a copy (clone) of the hash object. This can be used to
-                efficiently compute the digests of strings that share a common
-                initial substring.
-
-For example, to obtain the digest of the string 'Nobody inspects the
+ - update(data): Update the hash object with the bytes in data. Repeated calls
+                 are equivalent to a single call with the concatenation of all
+                 the arguments.
+ - digest():     Return the digest of the bytes passed to the update() method
+                 so far as a bytes object.
+ - hexdigest():  Like digest() except the digest is returned as a string
+                 of double length, containing only hexadecimal digits.
+ - copy():       Return a copy (clone) of the hash object. This can be used to
+                 efficiently compute the digests of datas that share a common
+                 initial substring.
+
+For example, to obtain the digest of the byte string 'Nobody inspects the
 spammish repetition':
 
     >>> import hashlib
@@ -130,14 +130,15 @@ def __get_openssl_constructor(name):
 
 def __py_new(name, data=b'', **kwargs):
     """new(name, data=b'', **kwargs) - Return a new hashing object using the
-    named algorithm; optionally initialized with data (which must be bytes).
+    named algorithm; optionally initialized with data (which must be
+    a bytes-like object).
     """
     return __get_builtin_constructor(name)(data, **kwargs)
 
 
 def __hash_new(name, data=b'', **kwargs):
     """new(name, data=b'') - Return a new hashing object using the named algorithm;
-    optionally initialized with data (which must be bytes).
+    optionally initialized with data (which must be a bytes-like object).
     """
     if name in {'blake2b', 'blake2s'}:
         # Prefer our blake2 implementation.

Lib/test/test_hashlib.py

@@ -568,8 +568,12 @@ def check_blake2(self, constructor, salt_size, person_size, key_size,
         self.assertRaises(ValueError, constructor, node_offset=-1)
         self.assertRaises(OverflowError, constructor, node_offset=max_offset+1)
 
+        self.assertRaises(TypeError, constructor, data=b'')
+        self.assertRaises(TypeError, constructor, string=b'')
+        self.assertRaises(TypeError, constructor, '')
+
         constructor(
-            string=b'',
+            b'',
             key=b'',
             salt=b'',
             person=b'',

Misc/NEWS.d/next/Library/2018-07-20-09-11-05.bpo-33729.sO6iTb.rst

@@ -0,0 +1 @@
+Fixed issues with arguments parsing in :mod:`hashlib`.

Modules/_blake2/blake2b_impl.c

@@ -47,10 +47,10 @@ typedef struct {
 #include "clinic/blake2b_impl.c.h"
 
 /*[clinic input]
-module _blake2b
-class _blake2b.blake2b "BLAKE2bObject *" "&PyBlake2_BLAKE2bType"
+module _blake2
+class _blake2.blake2b "BLAKE2bObject *" "&PyBlake2_BLAKE2bType"
 [clinic start generated code]*/
-/*[clinic end generated code: output=da39a3ee5e6b4b0d input=6893358c6622aecf]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=d47b0527b39c673f]*/
 
 
 static BLAKE2bObject *
@@ -66,13 +66,14 @@ new_BLAKE2bObject(PyTypeObject *type)
 
 /*[clinic input]
 @classmethod
-_blake2b.blake2b.__new__ as py_blake2b_new
-    string as data: object = NULL
+_blake2.blake2b.__new__ as py_blake2b_new
+    data: object(c_default="NULL") = b''
+    /
     *
-    digest_size: int(c_default="BLAKE2B_OUTBYTES") = _blake2b.blake2b.MAX_DIGEST_SIZE
-    key: Py_buffer = None
-    salt: Py_buffer = None
-    person: Py_buffer = None
+    digest_size: int(c_default="BLAKE2B_OUTBYTES") = _blake2.blake2b.MAX_DIGEST_SIZE
+    key: Py_buffer(c_default="NULL", py_default="b''") = None
+    salt: Py_buffer(c_default="NULL", py_default="b''") = None
+    person: Py_buffer(c_default="NULL", py_default="b''") = None
     fanout: int = 1
     depth: int = 1
     leaf_size: unsigned_long = 0
@@ -90,7 +91,7 @@ py_blake2b_new_impl(PyTypeObject *type, PyObject *data, int digest_size,
                     int fanout, int depth, unsigned long leaf_size,
                     unsigned long long node_offset, int node_depth,
                     int inner_size, int last_node)
-/*[clinic end generated code: output=65e732c66c2297a0 input=75ab5196b695adee]*/
+/*[clinic end generated code: output=65e732c66c2297a0 input=82be35a4e6a9daa2]*/
 {
     BLAKE2bObject *self = NULL;
     Py_buffer buf;
@@ -237,14 +238,14 @@ py_blake2b_new_impl(PyTypeObject *type, PyObject *data, int digest_size,
 }
 
 /*[clinic input]
-_blake2b.blake2b.copy
+_blake2.blake2b.copy
 
 Return a copy of the hash object.
 [clinic start generated code]*/
 
 static PyObject *
-_blake2b_blake2b_copy_impl(BLAKE2bObject *self)
-/*[clinic end generated code: output=c89cd33550ab1543 input=4c9c319f18f10747]*/
+_blake2_blake2b_copy_impl(BLAKE2bObject *self)
+/*[clinic end generated code: output=ff6acee5f93656ae input=e383c2d199fd8a2e]*/
 {
     BLAKE2bObject *cpy;
 
@@ -259,21 +260,21 @@ _blake2b_blake2b_copy_impl(BLAKE2bObject *self)
 }
 
 /*[clinic input]
-_blake2b.blake2b.update
+_blake2.blake2b.update
 
-    obj: object
+    data: object
     /
 
-Update this hash object's state with the provided string.
+Update this hash object's state with the provided bytes-like object.
 [clinic start generated code]*/
 
 static PyObject *
-_blake2b_blake2b_update(BLAKE2bObject *self, PyObject *obj)
-/*[clinic end generated code: output=a888f07c4cddbe94 input=3ecb8c13ee4260f2]*/
+_blake2_blake2b_update(BLAKE2bObject *self, PyObject *data)
+/*[clinic end generated code: output=010dfcbe22654359 input=ffc4aa6a6a225d31]*/
 {
     Py_buffer buf;
 
-    GET_BUFFER_VIEW_OR_ERROUT(obj, &buf);
+    GET_BUFFER_VIEW_OR_ERROUT(data, &buf);
 
     if (self->lock == NULL && buf.len >= HASHLIB_GIL_MINSIZE)
         self->lock = PyThread_allocate_lock();
@@ -293,14 +294,14 @@ _blake2b_blake2b_update(BLAKE2bObject *self, PyObject *obj)
 }
 
 /*[clinic input]
-_blake2b.blake2b.digest
+_blake2.blake2b.digest
 
-Return the digest value as a string of binary data.
+Return the digest value as a bytes object.
 [clinic start generated code]*/
 
 static PyObject *
-_blake2b_blake2b_digest_impl(BLAKE2bObject *self)
-/*[clinic end generated code: output=b13a79360d984740 input=ac2fa462ebb1b9c7]*/
+_blake2_blake2b_digest_impl(BLAKE2bObject *self)
+/*[clinic end generated code: output=a5864660f4bfc61a input=7d21659e9c5fff02]*/
 {
     uint8_t digest[BLAKE2B_OUTBYTES];
     blake2b_state state_cpy;
@@ -314,14 +315,14 @@ _blake2b_blake2b_digest_impl(BLAKE2bObject *self)
 }
 
 /*[clinic input]
-_blake2b.blake2b.hexdigest
+_blake2.blake2b.hexdigest
 
 Return the digest value as a string of hexadecimal digits.
 [clinic start generated code]*/
 
 static PyObject *
-_blake2b_blake2b_hexdigest_impl(BLAKE2bObject *self)
-/*[clinic end generated code: output=6a503611715b24bd input=d58f0b2f37812e33]*/
+_blake2_blake2b_hexdigest_impl(BLAKE2bObject *self)
+/*[clinic end generated code: output=b5598a87d8794a60 input=76930f6946351f56]*/
 {
     uint8_t digest[BLAKE2B_OUTBYTES];
     blake2b_state state_cpy;
@@ -335,10 +336,10 @@ _blake2b_blake2b_hexdigest_impl(BLAKE2bObject *self)
 
 
 static PyMethodDef py_blake2b_methods[] = {
-    _BLAKE2B_BLAKE2B_COPY_METHODDEF
-    _BLAKE2B_BLAKE2B_DIGEST_METHODDEF
-    _BLAKE2B_BLAKE2B_HEXDIGEST_METHODDEF
-    _BLAKE2B_BLAKE2B_UPDATE_METHODDEF
+    _BLAKE2_BLAKE2B_COPY_METHODDEF
+    _BLAKE2_BLAKE2B_DIGEST_METHODDEF
+    _BLAKE2_BLAKE2B_HEXDIGEST_METHODDEF
+    _BLAKE2_BLAKE2B_UPDATE_METHODDEF
     {NULL, NULL}
 };