What Pydantic features does Pyrefly need built-in support for?

[rchen152]

Starting a discussion about features provided by Pydantic's mypy plugin that we may need to special-case in Pyrefly. As a starting point, here's a list I came up with by diffing the mypy and Pyrefly outputs on Pydantic's mypy plugin tests:

Model configuration

Pydantic supports three ways to configure a model.

1. Class keywords [already supported]

from pydantic import BaseModel
class Model(BaseModel, frozen=True): ...

2. model_config attribute [unsupported]

from pydantic import BaseModel, ConfigDict
class Model(BaseModel):
    model_config = ConfigDict(frozen=True)

3. Nested Config class [unsupported]

from pydantic import BaseModel
class Model(BaseModel):
    class Config:
        frozen = True

Aliases

Pydantic provides many options for specifying the name by which a field is created versus the name under which it is stored.

1. alias [already supported]

from pydantic import BaseModel, Field
class Model(BaseModel):
    x: int = Field(alias='y')
Model(y=0)

2. validation_alias: essentially an alias for alias. When both are specified, alias is ignored. [unsupported]
3. validate_by_name, validate_by_alias: whether fields can be created by name or by alias, respectively [unsupported]

from pydantic import BaseModel, Field
class Model(BaseModel, validate_by_name=True, validate_by_alias=True):
    x: int = Field(alias='y')
Model(x=0)  # using name `x` is ok
Model(y=0)  # using alias `y` is also ok

4. alias_generator: a way to programmatically create aliases [unsupported]

from pydantic import BaseModel
class Model(BaseModel, alias_generator=lambda name: name + '_'):
    x: int
Model(x_=0)

Extra data

Pydantic lets you configure how extra data is handled. Importantly, the default is to ignore it, not error.

from pydantic import BaseModel
class Model(BaseModel, extra='ignore'):  # this is the default
    x: int
Model(x=0, y='ignored')

Strict mode

By default, Pydantic coerces data to the desired type, unless its “strict” mode is requested.

from pydantic import BaseModel, Field
class Model(BaseModel):
    x: int = Field(strict=False)  # this is the default
    y: int = Field(strict=True)
Model(x='0', y=1)  # ok, '0' is coerced to 0
Model(x='0', y='1')  # error, '1' is not an int

Unvalidated model construction

Pydantic provides a model_construct method that bypasses runtime validation. The mypy plugin still performs type validation.

from pydantic import BaseModel
class Model(BaseModel):
    x: int
Model.model_construct(x='oops')  # no runtime error, but a type checker should flag this

Misc.

Lastly, Pyrefly is missing a grab bag of smaller (in terms of expected implementation size) features:

1. Frozen models behave slightly differently from frozen dataclasses. Frozen-ness can be overridden in a subclass, and frozen can be set on individual fields.
2. Pydantic errors on fields missing type annotations.
3. The field_validator decorator transforms decorated instance methods into class methods. The model_validator decorator does this transformation in “before” and “wrap” modes.

[Viicos]

Thanks @rchen152 for putting this up.

Following our email exchange, here are a some notes on what could be supported by static type checkers regarding Pydantic behavior (and challenges in adding such support):

Configuration

• The Config class was used in V1 but is deprecated and will be removed in V3. I think it is reasonable to not support it at all.

• When using inheritance, configuration of a model is merged with the parent classes. However, things get complicated when multiple bases are present. Here is pseudo-code used to merge configuration together. For any class cls, with cls_kw being the class arguments:

  final_config = {}
  for base in cls.__bases__:
      final_config.update(getattr(base, 'model_config', {}))
  
  final_config.update(getattr(cls, 'model_config', {}))
  final_config.update(cls_kw)

  I don't believe that the class arguments overriding the model config is documented, but will probably not change. However, the merging of configuration from base classes is currently inconsistent as it doesn't follow the MRO and will likely change in V3 (tracked by model config in inheritance doesn't respect MRO pydantic/pydantic#9992). Which brings me to my next point

Change of behavior in V3

Pydantic behavior that affect static type checking may change from V2 to V3. The previous configuration is one example, but there could be others (different type coercing mechanisms, features to be removed/added in V3). Pyrefly could read the installed version from the installed package metadata, but I don't know if there are limitations with this approach

Type coercion

In non-strict mode, type coercion is performed on most supported types:

• The allowed inputs for each type is documented in https://docs.pydantic.dev/latest/api/standard_library_types, and in https://docs.pydantic.dev/latest/concepts/conversion_table/. However, some parts of this documentation are outdated, so I will need to properly review both of them.
• If using typing.Literal, strict behavior applies (e.g. Literal[True] only allows True not 1 or 'yes'), but may change in V3 (Make Literal behavior non-strict by default pydantic/pydantic#9991).
• The Strict metadata class can be used to mark specific fields as strict. Pydantic types such as StrictBool use this pattern.
• If plain, before, or wrap validators are used, it may change the allowed input types. Maybe the json_schema_input_type can be leveraged in this case.
• Type coercion may be tricky to support. It may require dynamically generated types for the __init__() signature (e.g. bool | Literal[0, 1, '0', 'off', 'f', 'false', 'n', 'no', '1', 'on', 't', 'true', 'y', 'yes', b'0', b'off', b'f', b'false', b'n', b'no', b'1', b'on', b't', b'true', b'y', b'yes'] for the bool type, or protocols with dunder methods).

[rchen152]

Thank you, @Viicos! Lots of great info here (cc @migeed-z, who is working on built-in support for Pydantic).

the merging of configuration from base classes is currently inconsistent as it doesn't follow the MRO and will likely change in V3

Huh, config merging not following the MRO is a bit surprising, but I'm pretty sure we can read the version of Pydantic that the user has installed if it continues to be a module-level string literal in version.py as I see here.

Type coercion

This definitely looks tricky to get right. This is a case where it would be really nice if some of this were expressible with dataclass_transform, which would help with both maintainability and the versioning problem. I'll look into this.

Also, I promised to point you at the start of our Pydantic implementation once we had something open-sourced, which we now do :) Tests live in pyrefly/lib/test/pydantic. Configuration options (just frozen for now) get extracted from ConfigDict here, stored as metadata on the class here, and applied to individual fields here.

[rchen152]

Quick update:

• Pyrefly now has experimental Pydantic support! We're collecting user feedback here.
• @migeed-z is working on putting together an examples repo to demo what's supported so far.