This update replaces instances of DecodeJson and DecodeJsonStr with UnmarshalJson and UnmarshalJsonStr in various relay handlers, enhancing code consistency and clarity in JSON processing. The changes improve maintainability and align with recent refactoring efforts in the codebase.
This update adds the IOCopyBytesGracefully function to the common package, which simplifies the process of copying response bodies in the OpenAI handlers. It enhances error handling and ensures proper resource management by encapsulating the logic for setting headers and writing response data. The OpenAI handlers have been refactored to utilize this new function, improving code clarity and maintainability.
This update standardizes the closure of HTTP response bodies across multiple stream handlers, enhancing error management and resource cleanup. The new method ensures that any errors during closure are handled gracefully, preventing potential request termination issues.
Changes:
- Replaced error returns with logging for response body copy failures to prevent early termination of the request.
- Ensured that the response body is closed properly after writing to the client.
- Added comments to clarify the handling of billing and error reporting after the response has been sent.
This update improves error handling and maintains resource management in the OpenAI handler.
- Changed the type of `Document` in `XinRerankResponseDocument` from `string` to `any` to accommodate various data types.
- Updated the `RerankHandler` to handle `Document` as `any`, ensuring proper assignment based on its actual type.
These modifications enhance the handling of document data, allowing for greater versatility in response structures.
- Updated TokenAuth middleware to handle requests for both `/v1beta/models/` and `/v1/models/`.
- Adjusted distributor middleware to recognize the new model path.
- Enhanced relay mode determination to include the new model path.
- Added route for handling POST requests to `/models/*path`.
These changes ensure compatibility with the new model API structure, improving the overall routing and authentication flow.
- Introduced `isNoThinkingRequest` and `trimModelThinking` functions to manage model names and thinking configurations.
- Updated `GeminiHelper` to conditionally adjust the model name based on the thinking budget and request settings.
- Refactored `ThinkingAdaptor` to streamline the integration of thinking capabilities into Gemini requests.
- Cleaned up commented-out code in `FetchUpstreamModels` for clarity.
These changes improve the handling of model configurations and enhance the adaptability of the Gemini relay system.
- Changed response handling from ObjectData to StringData for improved data processing.
- Ensured proper error logging in case of response handling failure.
- Added new handlers: AudioHelper, ImageHelper, EmbeddingHelper, and ResponsesHelper to manage respective requests.
- Updated ModelMappedHelper to accept request parameters for better model mapping.
- Enhanced error handling and validation across new handlers to ensure robust request processing.
- Introduced support for new relay formats in relay_info and updated relevant functions accordingly.
- Added status code mapping handling in GeminiHelper to reset status codes based on response.
- Removed redundant candidate check in GeminiTextGenerationHandler to simplify response processing.
CaIon