Required optional field has a default value of None after v1.7.0

[leovp]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.7.2
            pydantic compiled: False
                 install path: /home/leovp/Projects/pydantic/pydantic
               python version: 3.6.12 (default, Sep  5 2020, 11:22:16)  [GCC 7.5.0]
                     platform: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
     optional deps. installed: []

from typing import Optional

from pydantic import BaseModel


class Model(BaseModel):
    foo: int
    bar: Optional[int] = ...


m = Model(foo=1, bar=None)  # or Model(foo=1)
Model.parse_obj(m.dict(exclude_defaults=True))  # fails on pydantic v1.7.0+

It seems that pydantic thinks bar (a "required optional" field) has a default value of None and serializes the model without this key, which makes it non-deserializable. This was not the case on versions 1.5.x and 1.6.x, so I think it's a regression. (Or maybe it was never a guarantee).

According to git bisect the potential regression was introduced in commit 4bc4230.

[leovp]

https://github.com/samuelcolvin/pydantic/blob/31bc2435d7968fdf8d1f0c5c67f0f851c1bef54e/pydantic/main.py#L789

Alright, I haven't looked really closely, but here's my guess: getattr here can never fallback to _missing, since ModelField.default is always there and has a value of None unless set explicitly:
https://github.com/samuelcolvin/pydantic/blob/31bc2435d7968fdf8d1f0c5c67f0f851c1bef54e/pydantic/fields.py#L255

UPD1: And here's where "bar" is created and gets the default value of None: https://github.com/samuelcolvin/pydantic/blob/31bc2435d7968fdf8d1f0c5c67f0f851c1bef54e/pydantic/fields.py#L308-L310

UPD2: Is it a good idea to check field.field_info.default instead of field.default? In this particular case it has a correct value of Ellipsis, and you could special-case it while serialising the model.

[PrettyWood]

Hello @leovp and thanks for reporting :)
You're absolutely right! IMO the good fix would be to set default = Ellipsis when required
Would you like to make a PR for this? If you want I can do it as well tonight for a v1.7.3.
Please tell me.
Cheers

[leovp]

@PrettyWood
It's 11PM where I live, so please go ahead :)