FastAPI (0.101.0) Broke OpenAPI naming

[syniex]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from fastapi import FastAPI
from pydantic import BaseModel,Field
import typing


app = FastAPI()


class JustLovelyName(BaseModel):
    name: typing.Union[str,None] = None


@app.get("/output")
def output() -> JustLovelyName:
    return  JustLovelyName(name="Hello World")

@app.post("/input")
def two(name:JustLovelyName) -> JustLovelyName:
    return name


if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8002)

Description

OpenAPI for JustLovelyName get suffix of Input and output.

This happens when a Model has a default value (apparently None or Field(None))

In version 0.100.1 it will generate only JustLovelyName.
In version 0.101.0 it will generate both JustLovelyNameOutput and JustLovelyNameInput.

removing the usage as input or removing the default: None will make it only JustLovelyName.

Operating System

Linux

Operating System Details

LXC container

FastAPI Version

0.101.0

Pydantic Version

2.1.1

Python Version

3.11.2

Additional Context

[commonism]

related #10039 #10041
The models have different requirements for their properties as well wrt. required/nullable.

[Harry-Lees]

I added the above code as a test

from typing import Union
from fastapi import FastAPI
from fastapi.testclient import TestClient
from pydantic import BaseModel

app = FastAPI()

class JustLovelyName(BaseModel):
    name: Union[str, None] = None

@app.get("/output")
def output() -> JustLovelyName:
    return JustLovelyName(name="Hello World")

@app.post("/input")
def two(name: JustLovelyName) -> JustLovelyName:
    return name

client = TestClient(app)

def test_get_output():
    response = client.get("/output")
    assert response.status_code == 200, response.text

def test_post_input():
    response = client.post("/input", json={"name": "Hello World"})
    assert response.status_code == 200, response.text

def test_openapi():
    response = client.get("/openapi.json")
    assert response.status_code == 200, response.text
    assert response.json() == {...}

to run git bisect which revealed that this issue was introduced in 19a2c3b.

[sebastianvitterso]

@tiangolo Is there any chance we could be allowed to disable this new functionality?

This change made our api generator (which is the one recommended by FastAPI docs) suddenly delete all our generated model files, and replace each one with ModelInput and ModelOutput, which would make us have to suddenly "care" about which way the model is used. We're currently using the models directly both as input and output, and want them to work interchangably.

[dmontagu]

@tiangolo I'd be happy to brainstorm with you about how to best handle this, whether in fastapi or pydantic.

[tiangolo]

Yep, I just added a feature to disable it in FastAPI 0.102.0 🎉

The new docs are here: https://fastapi.tiangolo.com/how-to/separate-openapi-schemas/

[commonism]

Can be reproduced with pydantic only.

@Kludex any ideas how to work this?

import io

import yaml

from pydantic import BaseModel
from pydantic.json_schema import models_json_schema


class Pet(BaseModel):
    name: str
    weight: int | None
    color: str | None = None

data = models_json_schema([(Pet,"serialization"),(Pet,"validation")])
f = io.StringIO()
for d in data:
    for k,v in d.items():
        print(yaml.dump(v, f, indent=4))
f.seek(0)
print(f.read())

$ref: '#/$defs/PetOutput'
$ref: '#/$defs/PetInput'
PetInput:
    properties:
        color:
            anyOf:
            -   type: string
            -   type: 'null'
            default: null
            title: Color
        name:
            title: Name
            type: string
        weight:
            anyOf:
            -   type: integer
            -   type: 'null'
            title: Weight
    required:
    - name
    - weight
    title: Pet
    type: object
PetOutput:
    properties:
        color:
            anyOf:
            -   type: string
            -   type: 'null'
            default: null
            title: Color
        name:
            title: Name
            type: string
        weight:
            anyOf:
            -   type: integer
            -   type: 'null'
            title: Weight
    required:
    - name
    - weight
    - color
    title: Pet
    type: object

[dmontagu]

@commonism your example tweaked to use a custom json schema generator to only use the logic for mode='validate' for determining whether fields are required:

import io

import yaml
from pydantic_core import core_schema

from pydantic import BaseModel
from pydantic.json_schema import models_json_schema, GenerateJsonSchema


class Pet(BaseModel):
    name: str
    weight: int | None
    color: str | None = None


class MyGenerateJsonSchema(GenerateJsonSchema):
    def field_is_required(
        self,
        field: core_schema.ModelField | core_schema.DataclassField | core_schema.TypedDictField,
        total: bool,
    ) -> bool:
        if field['type'] == 'typed-dict-field':
            return field.get('required', total)
        else:
            return field['schema']['type'] != 'default'


data = models_json_schema([(Pet, "serialization"), (Pet, "validation")], schema_generator=MyGenerateJsonSchema)
f = io.StringIO()
for d in data:
    for k, v in d.items():
        print(yaml.dump(v, f, indent=4))
f.seek(0)
print(f.read())

With this, you only get one Pet model, not Pet-Input and Pet-Output.

[commonism]

I've had no hands on pydantic.json_schema.GenerateJsonSchema so far, but I feel honored you think so.
Being able to adjust the behavior is great, but I do not think it is intended to adjust fastapis GenerateJsonSchema internals, so it does not help outside our synthetic example.

I understand there is a distinction in having different personalities for the same model wrt. to validation/serialization, but currently fastapi is leaking these internal personas to the schema (schemazophrenia?), which is very unfortunate as it … basically invalidates everything which has been using fastapi/pydantic.

It messes up the description documents.
It breaks all existing clients for existing services. The models are not even interchangeable - you can't use a response model as request body.
The models even have different behavior - in the example the Output models color is not optional any longer.
And I do not see the benefit.

I'd prefer to continue this discussion in an issue tracker as to me discussions do not work as there is no continuation line.
You can respond to any existing answer or start a new answer - impossible to keep track of.

[commonism]

Issue pydantic/pydantic#7209

[tiangolo]

Yep, as @dmontagu says, this is actually the correct behavior, it's more correct, because things work differently for input and output, when you think about them from the point of view of the client. Anyway, you can check the rest of my answer here: #10019 (comment)

The new feature to allow disabling this behavior is in FastAPI 0.102.0 🎉

You can check the new docs here: https://fastapi.tiangolo.com/how-to/separate-openapi-schemas/

[jcphlux]

This works in most cases. The only issue I have is computed fields are not included in the model. I would be nice to choose between using the input or output model to use in the schema.

[tiangolo]

Yep, so this is actually intentional. 😅

There was a lot of work from the Pydantic side and here to make it work this way... let me explain.

The idea is that an input model can have fields that are not required if they have default values.

But if you return that same model, because it will always have a value (if you don't provide one it will have the default value), then the clients could expect that it will always have a value. Nevertheless, this was not reflected in OpenAPI up to now. The way to do that is by saying that the response model with a field that has a default value is required.

This is in particular useful if you are autogenerating client code/SDKs, because now that code will be more correct. And the developers using that client code/SDKs won't have to add a bunch of checks to see if the value is provided or not, they will be able to simplify their code and know that there will always be a value, because the type information will include that.

Now, given that several would prefer to keep the old behavior, probably/hopefully just for a bit before upgrading the client code, I added a feature to allow telling FastAPI to not separate models with separate_input_output_schemas=False, internally it uses the new features in Pydantic for that.

It's available in FastAPI 0.102.0, just released, and there are new docs for that here: https://fastapi.tiangolo.com/how-to/separate-openapi-schemas/ 🎉

[commonism]

Hi,

This is in particular useful if you are autogenerating client code/SDKs, because now that code will be more correct.

In my example. if a pet without color is received, the .color property is None by the default value of the model.
How does making the color property required and forcing serialization of the None value make it more correct?

Can you provide an example?

And the developers using that client code/SDKs won't have to add a bunch of checks to see if the value is provided or not,

It's a default value, it does not have to be provided.
The value of a property which is unset is the default value.
A default value does not not have to be transferred/required.

they will be able to simplify their code and know that there will always be a value,

None - a quite common default value for optional fields, is something you'll have to test for even when using -Output models.

because the type information will include that.

Has been included in the type information before - default value definition.

pydantic mixes "has a default value" with "is not required", it is not possible to have something not-required without setting a default value.
In pydantic "not required" is "has a default value" .
The serialization mode schema requires non required fields - making use of the default value.
Therefore - the serialization mode includes fields with default values as required (e.g. PetOutput.color).
Redundant information is transferred by transferring default values already defined in the schema.

For validation (of non-required-has-a-default-value serialized fields) - if the value is skipped - it has a default value defined.

Where is the benefit of defining a new model with the non-required field defined as required to validate the serialized default value.
It's not even required to serialize default values (special case of discriminated unions aside).

I do not see having non-interchangeable model definitions for "Input" and "Output" simplifying a use-case.
These models end up as classes. Operations would have to accept …Input | …Output models as arguments to make them interchangeable.
Models can be nested.

[fredrikaverpil]

This entire change that got introduced in FastAPI 0.101.0 seems extremely fragile to me.

You may refactor your code in your FastAPI service and not even realize that one schema name might be removed and two new ones will be created in its place. This is very unexpected and fragile behavior.

If the above is true, there will be a high probability something breaks for e.g. clients generated from the OpenAPI schema.

[fredrikaverpil]

@tiangolo I am using app = FastAPI(separate_input_output_schemas=False) but I still see the split input/output schemas in my Swagger documentation and my TS client codegen also generates objects with Input/Output. So I think this argument doesn't fully work as expected. Do you see this on your end too?

Because of this, I had to downgrade to FastAPI 0.100.1 here.

Also, see my comment here on that this OpenAPI codegen change seems extremely fragile to me.

[fredrikaverpil]

I found the problem; I'm adding customizations to the openapi schema so I'm following the instructions in here: https://fastapi.tiangolo.com/how-to/extending-openapi

This mentions the fastapi.openapi.utils.get_openapi function call, that now we also need to pass this variable into. The docs does not mention it.

TL;DR you must pass separate_input_output_schemas=False into both the app = FastAPI() call and into the fastapi.openapi.utils.get_openapi call.