Skip to content

API,BUG,DEP: treat trailing comma as a tuple and thus a structured dtype.#25434

Merged
ngoldbaum merged 1 commit intonumpy:mainfrom
mhvk:dtype-from-string-with-trailing-comma
Dec 21, 2023
Merged

API,BUG,DEP: treat trailing comma as a tuple and thus a structured dtype.#25434
ngoldbaum merged 1 commit intonumpy:mainfrom
mhvk:dtype-from-string-with-trailing-comma

Conversation

@mhvk
Copy link
Contributor

@mhvk mhvk commented Dec 19, 2023

Previously, np.dtype("i") and np.dtype("i,") were treated as identical, even though np.dtype("i,i") creates a structured dtype with two fields. Now, np.dtype("i,") will create a structured dtype as well, with a single field.

At the same time, the use of np.dtype("(2)i,") to create a subarray with 2 elements is deprecated; one should use np.dtype("(2,)i") or np.dtype("2i") instead. This is consistent with np.dtype("(2)i") raising an exception; in that sense, the previously allowed usage was a bug.

fixes #25429

@mhvk mhvk added this to the 2.0.0 release milestone Dec 19, 2023
@mhvk mhvk force-pushed the dtype-from-string-with-trailing-comma branch from 6bbb120 to 1c0c76c Compare December 19, 2023 20:44
@seberg seberg added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Dec 19, 2023
@mhvk mhvk changed the title API,BUG: treat trailing comma as a tuple and thus a structured dtype. API,BUG,DEP: treat trailing comma as a tuple and thus a structured dtype. Dec 19, 2023
@mhvk mhvk force-pushed the dtype-from-string-with-trailing-comma branch 2 times, most recently from 1b11192 to 41031e0 Compare December 19, 2023 21:12
@mhvk
Copy link
Contributor Author

mhvk commented Dec 19, 2023

Added a release note...

@mhvk mhvk force-pushed the dtype-from-string-with-trailing-comma branch from 41031e0 to 3d7540e Compare December 19, 2023 21:55
@seberg seberg removed the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Dec 20, 2023
Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM, just nits, this is OK to put in to me.

If someone has an idea to shorten the release note, I wouldn't mind.
Otherwise, I am ambivalent towards the deprecation, but at least if it is tedious to make (2)i (without any comma, I think it takes a different code path!) work, neither do I care.

I would appreciate cleaning up the C-calling code a bit (cache _commastring not the module).

(To me the Python side parsing code looks a bit overly complicated, but not a problem here...)

}
if (PyList_GET_SIZE(listobj) == 1) {
res = _convert_from_any(PyList_GET_ITEM(listobj, 0), align);
else if (PyList_Check(parsed) && PyList_GET_SIZE(parsed) >= 1) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
else if (PyList_Check(parsed) && PyList_GET_SIZE(parsed) >= 1) {
else if (PyList_Check(parsed)) {

I suppose length 0 is impossible here anyway? I suspect we could remove the size check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the length checks were mainly meant to guard against silly mistakes in the python code. My sense would be to leave it in.

…ype.

Previously, `np.dtype("i")` and `np.dtype("i,")` were treated as
identical, even though `np.dtype("i,i")` creates a structured dtype
with two fields.  Now, `np.dtype("i,")` will create a structured dtype
as well, with a single field.

At the same time, the use of `np.dtype("(2)i,")` to create a subarray
with 2 elements is deprecated; one should use `np.dtype("(2,)i")` or
`np.dtype("2i")` instead.  This is consistent with `np.dtype("(2)i")`
raising an exception.
@mhvk mhvk force-pushed the dtype-from-string-with-trailing-comma branch from 3d7540e to 86b39ab Compare December 20, 2023 15:05
Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM now. Will wait a bit in case someone has a comment or just feels like pressing the green button :).

@ngoldbaum
Copy link
Member

Thanks @mhvk!

@ngoldbaum ngoldbaum merged commit cad2c17 into numpy:main Dec 21, 2023
@mhvk mhvk deleted the dtype-from-string-with-trailing-comma branch December 21, 2023 17:43
tylerjereddy added a commit to tylerjereddy/scipy that referenced this pull request Dec 23, 2023
* Fixes scipy#19737

* the deprecation is obviously coming from numpy/numpy#25434; I
picked one of the alternatives Marten suggested there and it passes
locally on x86_64 Linux with NumPy `main` and `1.26.2`

[skip cirrus]
j-bowhay pushed a commit to scipy/scipy that referenced this pull request Dec 23, 2023
* Fixes #19737

* the deprecation is obviously coming from numpy/numpy#25434; I
picked one of the alternatives Marten suggested there and it passes
locally on x86_64 Linux with NumPy `main` and `1.26.2`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: dtype() handles single-length comma-separated fields as scalar

3 participants