feat: multi-pass type name resolution#2213
feat: multi-pass type name resolution#2213mromaszewicz wants to merge 6 commits intooapi-codegen:mainfrom
Conversation
|
Hello! The TMF spec defines The requestBody variant has multiple content types:
These are all valid, standardized media types. The problem is in contentTypeSuffix() it uses strings.Contains(ct, "json") case strings.Contains(ct, "json"):
return "JSON"This causes a loop in resolveCollisions. I think using exact matches for the common media types instead of substring matching, func contentTypeSuffix(ct string) string {
if ct == "" {
return ""
}
ct = strings.ToLower(ct)
switch ct {
case "application/json":
return "JSON"
case "application/xml", "text/xml":
return "XML"
case "application/x-www-form-urlencoded", "multipart/form-data":
return "Form"
case "text/plain":
return "Text"
case "application/octet-stream":
return "Binary"
case "application/yaml", "application/x-yaml":
return "YAML"
default:
return mediaTypeToCamelCase(ct)
}
}Additionally, I think strategyContextSuffix should track whether a context suffix has already been applied to prevent the loop. What do you think about that? Thank you for your work! |
|
Thank you for finding this. i will add it to my test cases. I'm trying my best to make these changes without breaking backward compatibility, and if I CamelCase-convert the full media type, it's going to change the code generation output for existing users who might not expect their JSON suffix to change to MergePatchJSON or something. I'm back-porting these fixes from an experiment that I am doing, and I think the only thing I can really do is delegate the choice of converting the full media type to a short name to the user via configuration options. Look here at the documentation for Whenever I make what I think are simple test cases that are representative, people in the real world have more complex usage :) I never really thought about returning multiple JSON models from a single API. |
|
Ah okay, the Trying to fix that issue from above highlighted more issues I briefly described in my PR on your fork here: mromaszewicz#1 |
|
Thanks for doing that. This code, as is, should resolve your conflict by appending numerical suffixes to the end after all the collisions. I think I'm doing the multi-pass wrong. Each strategy needs to be applied globally over all schemas before trying the next one. |
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@mitsaucepls , please try this latest patch on your schema. I've added your specific example as a test case, and it now works. When, after all rounds of conflict resolution fail, the numeric fallback catches this, and you end up with The problem was that you can't try conflict resolution strategies on each conflict group, you must run each strategy globally until it makes no more changes, before moving onto the next one. When you rename things in one conflict group, you may actually rename them into conflict with another schema. This problem is very, very complicated, which is why I avoided it until now. Thanks for those PR's you are sending, but they're not the right fix for the issue because they can put the new names into conflict with other stuff. This PR doesn't contain the short names map, that will be a separate pull request which will let you avoid the numerical suffixes and replace them with more meaningful content type abbreviations. |
7773945 to
fb9c473
Compare
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fb9c473 to
cad170e
Compare
|
Hey, it definitely generates now! But there are still errors because of undefined types. I found that helper types are not generated and that causes the undefined type behavior. diff --git a/pkg/codegen/codegen.go b/pkg/codegen/codegen.go
index 279ed9d..437af32 100644
--- a/pkg/codegen/codegen.go
+++ b/pkg/codegen/codegen.go
@@ -695,6 +695,7 @@ func GenerateTypesForResponses(t *template.Template, responses openapi3.Response
}
types = append(types, typeDef)
+ types = append(types, goType.AdditionalTypes...)
}
}
return types, nilWithout knowing I can imagine other GenerateTypesFor... functions are also affected, just not in my specific case. |
|
Thanks @mitsaucepls . You found another old issue. Neither Responses or RequestBodies handle the additional types, so they miss generating some models. |
|
Please try it again. If that spec continues to have problems, could you attach it here? I couldn't find the OpenAPI file to download. |
|
I just realized I was on main for my last testing... sorry :D This is the OpenAPI Yaml file Ive been using. TMF622-ProductOrdering-v5.0.0.oas.yaml alternatively downloadable here but it requires an account. This is the config.yaml I used so far: package: api
generate:
echo-server: true
strict-server: true
models: true
output-options:
resolve-type-name-collisions: true
output: gen.api.goWhen using the fix/name-collisions branch there are still some errors. But fewer than before! |
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0d74374 to
8107da4
Compare
|
I'm going to split that AdditionalTypes fix out of this PR and into a new one, with a new issue, because they are independent problems. |
8107da4 to
98dfe03
Compare
|
Man, that spec you attached is a doozy. It exposed a few more issues. Now, it generates cleanly with the latest update. |
9ae0501 to
50fd1a4
Compare
I've been meaning to use this approach for a long time, because the attempts at avoiding type collisions via structure suffixes or prefixes work sporadically, at best. Conflict resolution is fundamentally a global problem, not a local problem when doing recursive traversal, so this PR splits the code generation into two parts. First, the OAPI document structure is traversed, and all the schemas that we generate are gathered up into a list of candidates, then we do global conflict resolution across the space of all schemas. This allows us to preserve the functionality of things affected by schema name - `$ref`, `required` properties, and so forth. This fixes issue oapi-codegen#1474 (client response wrapper type colliding with a component schema of the same name and improves issue oapi-codegen#200 handling (same name across schemas, parameters, responses, requestBodies, headers). The new system is gated behind the existing `resolve-type-name-collisions` output option. When disabled, behavior is unchanged, oapi-codegen exits with an error. This flag is default false, so there is no behavior change to oapi-codegen unless it's specified. All current test files regenerate without any differences. Added a comprehensive test which reproduces the scenarios in all the PR's and Issues below, and adds a few more, to make sure that references to renamed targets are also correct.. Key changes: - gather.go: walks entire spec collecting schemas with location metadata - resolve_names.go: assigns unique names via context suffix, per-schema disambiguation, and numeric fallback strategies - Component schemas are privileged and keep bare names on collision - Client response wrapper types now participate in collision detection - Removed ComponentType/DefinedComp from Schema struct - Removed FixDuplicateTypeNames and related functions from utils.go Obsoletes issues: - oapi-codegen#1474 Schema name vs client wrapper (CreateChatCompletionResponse) - oapi-codegen#1713 Schema name vs client wrapper (CreateBlueprintResponse) - oapi-codegen#1450 Schema name vs client wrapper (DeleteBusinessResponse) - oapi-codegen#2097 Path response type vs schema definition (Status) - oapi-codegen#255 Endpoint path vs response type (QueryResponse) - oapi-codegen#899 Duplicate types from response wrapper vs schema (AccessListResponse) - oapi-codegen#1357 Schema vs operationId response (ListAssistantsResponse, OpenAI spec) - oapi-codegen#254 Cross-section: requestBodies vs schemas (Pet) - oapi-codegen#407 Cross-section: requestBodies vs schemas (myThing) - oapi-codegen#1881 Cross-section: requestBodies with multiple content types Obsoletes PRs: - oapi-codegen#292 Parameter structures params postfix (superseded by context suffix) - oapi-codegen#1005 Fix generate equals structs (superseded by multi-pass resolution) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> EOF )
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add test case from oapi-codegen-exp#14: an inline response object whose properties $ref component schemas with x-go-type: string. In the experimental rewrite (libopenapi), this caused duplicate type declarations because libopenapi copies extensions from $ref targets. V2 (kin-openapi) handles this correctly, but the test guards against future regressions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a component response has multiple JSON content types (e.g., application/json, application/json-patch+json, application/merge-patch+json), three bugs produced uncompilable code: 1. Duplicate inline type declarations: GenerateGoSchema was called with a path of [operationId, responseName] (or [responseName] in the component phase), which doesn't include the content type. When multiple content types share oneOf schemas with inline elements, they produce identically-named AdditionalTypes. If the schemas differ, the result is conflicting declarations; if they're the same, duplicate declarations. Fix: include a mediaTypeToCamelCase(contentType) segment in the schema path when jsonCount > 1, giving each content type's inline types unique names. This is guarded by jsonCount > 1 for backward compatibility. 2. Undefined types in unmarshal code: GetResponseTypeDefinitions used RefPathToGoType to resolve $ref paths like #/components/responses/200Resource_Patch, but without content type context. resolvedNameForComponent fell into a non-deterministic prefix match, returning an arbitrary per-content-type base name that didn't match any defined type when mediaTypeToCamelCase was appended. Fix: add resolvedNameForRefPath helper that parses the $ref path and delegates to resolvedNameForComponent with the specific content type for exact matching. 3. Mismatched types in response struct fields: same root cause as bug 2 — the struct field types were derived from the same non-deterministic resolution path. Additionally, promote AdditionalTypes from GenerateTypesForResponses and GenerateTypesForRequestBodies to the top-level type list, matching the existing pattern in GenerateTypesForSchemas. Without this, inline types (e.g., oneOf union members, nested objects with additionalProperties) defined inside response/requestBody schemas were silently dropped from the output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
50fd1a4 to
dd19a4e
Compare
I've been meaning to use this approach for a long time, because the attempts at avoiding type collisions via structure suffixes or prefixes work sporadically, at best.
Conflict resolution is fundamentally a global problem, not a local problem when doing recursive traversal, so this PR splits the code generation into two parts. First, the OAPI document structure is traversed, and all the schemas that we generate are gathered up into a list of candidates, then we do global conflict resolution across the space of all schemas. This allows us to preserve the functionality of things affected by schema name -
$ref,requiredproperties, and so forth.This fixes issue #1474 (client response wrapper type colliding with a component schema of the same name and improves issue #200 handling (same name across schemas, parameters, responses, requestBodies, headers).
The new system is gated behind the existing
resolve-type-name-collisionsoutput option. When disabled, behavior is unchanged, oapi-codegen exits with an error. This flag is default false, so there is no behavior change to oapi-codegen unless it's specified. All current test files regenerate without any differences.Added a comprehensive test which reproduces the scenarios in all the PR's and Issues below, and adds a few more, to make sure that references to renamed targets are also correct..
Key changes:
Obsoletes issues:
Obsoletes PRs:
EOF
)