Eino中的模型封装，以deepseek为例

deepseek 中模型配置参数：

TopP&Temperature
在 DeepSeek 模型配置中，Top-p（又常写作 top_p 或 “核采样 / nucleus sampling”）是一个 0–1 之间的浮点参数，用来动态限制模型每一步可选词的范围，从而控制生成文本的多样性与确定性。其含义和工作方式可概括为：
数值含义
• Top-p = 1：累积概率 100%，等价于不限制，所有词都可能被选到，生成最“放飞”。
• Top-p = 0.8：只保留累积概率前 80% 的高频词，过滤掉尾部 20% 的低概率词。
• Top-p → 0：候选集合极度缩小，趋近于每次都选最高概率词，输出最保守、最确定。
与温度（temperature）的配合
温度先把原始概率分布做“软化”或“锐化”，Top-p 再在此基础上进行“二次截断”。
• 高温 + 高 Top-p：极度随机，适合创意写作、头脑风暴。
• 低温 + 低 Top-p：几乎确定，适合技术文档、代码补全。

典型场景推荐值：

PresencePenalty&FrequencyPenalty

把 presence_penalty 调高（如 1.2），模型会尽量避免重复任何已用过的词，可能换来更多生僻词或跳跃式话题。
把 frequency_penalty 调高（如 1.0），模型仍可使用出现过的词，但会抑制“三遍四遍”式啰嗦，适合减少口水话而不破坏连贯性。

Stop
stop 是一个字符串或字符串列表，用来告诉模型：“一旦生成的文本里出现这里面的任意一个字符串，就立刻停止继续生成。”换句话说，它是人为设置的“终止标识符”

空值或空列表等价于“不设置任何额外停止条件”，模型会一直生成到 max_tokens 或 EOS token。

type ChatModelConfig struct {
    // APIKey is your authentication key
    // Required
    APIKey string `json:"api_key"`

    // Timeout specifies the maximum duration to wait for API responses
    // Optional. Default: 5 minutes
    Timeout time.Duration `json:"timeout"`

    // BaseURL is your custom deepseek endpoint url
    // Optional. Default: https://api.deepseek.com/
    BaseURL string `json:"base_url"`

    // The following fields correspond to DeepSeek's chat API parameters
    // Ref: https://api-docs.deepseek.com/api/create-chat-completion

    // Model specifies the ID of the model to use
    // Required
    Model string `json:"model"`

    // MaxTokens limits the maximum number of tokens that can be generated in the chat completion
    // Range: [1, 8192].
    // Optional. Default: 4096
    MaxTokens int `json:"max_tokens,omitempty"`

    // Temperature specifies what sampling temperature to use
    // Generally recommend altering this or TopP but not both.
    // Range: [0.0, 2.0]. Higher values make output more random
    // Optional. Default: 1.0
    Temperature float32 `json:"temperature,omitempty"`

    // TopP controls diversity via nucleus sampling
    // Generally recommend altering this or Temperature but not both.
    // Range: [0.0, 1.0]. Lower values make output more focused
    // Optional. Default: 1.0
    TopP float32 `json:"top_p,omitempty"`

    // Stop sequences where the API will stop generating further tokens
    // Optional. Example: []string{"\n", "User:"}
    Stop []string `json:"stop,omitempty"`

    // PresencePenalty prevents repetition by penalizing tokens based on presence
    // Range: [-2.0, 2.0]. Positive values increase likelihood of new topics
    // Optional. Default: 0
    PresencePenalty float32 `json:"presence_penalty,omitempty"`

    // ResponseFormat specifies the format of the model's response
    // Optional. Use for structured outputs
    ResponseFormatType ResponseFormatType `json:"response_format_type,omitempty"`

    // FrequencyPenalty prevents repetition by penalizing tokens based on frequency
    // Range: [-2.0, 2.0]. Positive values decrease likelihood of repetition
    // Optional. Default: 0
    FrequencyPenalty float32 `json:"frequency_penalty,omitempty"`
}

ChatModel 定义：这里引用了第三方封装好的deepseek客户端， 并封装了config配置和工具调用tool

type ChatModel struct {
    cli  *deepseek.Client
    conf *ChatModelConfig

    tools      []deepseek.Tool
    rawTools   []*schema.ToolInfo
    toolChoice *schema.ToolChoice
}

func NewChatModel(_ context.Context, config *ChatModelConfig) (*ChatModel, error) {
    if len(config.APIKey) == 0 {
        return nil, fmt.Errorf("API key is required")
    }
    if len(config.Model) == 0 {
        return nil, fmt.Errorf("model is required")
    }

    var opts []deepseek.Option
    if config.Timeout > 0 {
        opts = append(opts, deepseek.WithTimeout(config.Timeout))
    }
    if len(config.BaseURL) > 0 {
        opts = append(opts, deepseek.WithBaseURL(config.BaseURL))
    }

    cli, err := deepseek.NewClientWithOptions(config.APIKey, opts...)
    if err != nil {
        return nil, err
    }
    return &ChatModel{cli: cli, conf: config}, nil
}

deepseek 的Tool 类型主要定义调用函数的名称， 描述和参数信息等

// FunctionParameters defines the parameters for a function.
type FunctionParameters struct {
    Type       string                 `json:"type"`                 // The type of the parameters, e.g., "object" (required).
    Properties map[string]interface{} `json:"properties,omitempty"` // The properties of the parameters (optional).
    Required   []string               `json:"required,omitempty"`   // A list of required parameter names (optional).
}

// Function defines the structure of a function tool.
type Function struct {
    Name        string              `json:"name"`                 // The name of the function (required).
    Description string              `json:"description"`          // A description of the function (required).
    Parameters  *FunctionParameters `json:"parameters,omitempty"` // The parameters of the function (optional).
}

// Tool defines the structure for a tool.
type Tool struct {
    Type     string   `json:"type"`     // The type of the tool, e.g., "function" (required).
    Function Function `json:"function"` // The function details (required).
}

openAI api 接口返回示例：

{
  "id": "chatcmpl-1234567890abcdef",
  "object": "chat.completion",
  "created": 1699989180,
  "model": "gpt-4o-mini-2024-07-18",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 9,
    "total_tokens": 18
  }
}

deepseek及其他主流模型的会话类型分一次性完成回答和流式回答：

// BaseChatModel defines the basic interface for chat models.
// It provides methods for generating complete outputs and streaming outputs.
// This interface serves as the foundation for all chat model implementations.
//
//go:generate  mockgen -destination ../../internal/mock/components/model/ChatModel_mock.go --package model -source interface.go
type BaseChatModel interface {
    Generate(ctx context.Context, input []*schema.Message, opts ...Option) (*schema.Message, error)
    Stream(ctx context.Context, input []*schema.Message, opts ...Option) (
        *schema.StreamReader[*schema.Message], error)
}

这两种会话类型对应BaseChatModel 接口的Generate 和Stream 两个方法， 故enio 在封装deepseek模型时需要实习这两个接口方法。

func (cm *ChatModel) Generate(ctx context.Context, in []*schema.Message, opts ...model.Option) (outMsg *schema.Message, err error) {
    defer func() {
        if err != nil {
            callbacks.OnError(ctx, err)
        }
    }()

    req, cbInput, err := cm.generateRequest(ctx, in, opts...)

    ctx = callbacks.OnStart(ctx, cbInput)

    resp, err := cm.cli.CreateChatCompletion(ctx, req)
    if err != nil {
        return nil, fmt.Errorf("failed to create chat completion: %w", err)
    }

    if len(resp.Choices) == 0 {
        return nil, fmt.Errorf("received empty choices from DeepSeek API response")
    }

    for _, choice := range resp.Choices {
        if choice.Index != 0 {
            continue
        }

        outMsg = &schema.Message{
            Role:    toMessageRole(choice.Message.Role),
            Content: choice.Message.Content,
            // TODO: tool call
            ResponseMeta: &schema.ResponseMeta{
                FinishReason: choice.FinishReason,
                Usage:        toEinoTokenUsage(&resp.Usage),
            },
        }
        if len(choice.Message.ReasoningContent) > 0 {
            SetReasoningContent(outMsg, choice.Message.ReasoningContent)
        }

        break
    }

    if outMsg == nil {
        return nil, fmt.Errorf("invalid response format: choice with index 0 not found")
    }

    callbacks.OnEnd(ctx, &model.CallbackOutput{
        Message:    outMsg,
        Config:     cbInput.Config,
        TokenUsage: toCallbackUsage(outMsg.ResponseMeta.Usage),
    })

    return outMsg, nil
}



func (cm *ChatModel) Stream(ctx context.Context, in []*schema.Message, opts ...model.Option) (outStream *schema.StreamReader[*schema.Message], err error) {
    defer func() {
        if err != nil {
            callbacks.OnError(ctx, err)
        }
    }()
    req, cbInput, err := cm.generateStreamRequest(ctx, in, opts...)

    ctx = callbacks.OnStart(ctx, cbInput)

    stream, err := cm.cli.CreateChatCompletionStream(ctx, req)
    if err != nil {
        return nil, fmt.Errorf("failed to create chat stream completion: %w", err)
    }

    sr, sw := schema.Pipe[*model.CallbackOutput](1)
    go func() {
        defer func() {
            panicErr := recover()
            _ = stream.Close()

            if panicErr != nil {
                _ = sw.Send(nil, newPanicErr(panicErr, debug.Stack()))
            }

            sw.Close()
        }()

        var lastEmptyMsg *schema.Message

        for {
            chunk, chunkErr := stream.Recv()
            if errors.Is(chunkErr, io.EOF) {
                if lastEmptyMsg != nil {
                    sw.Send(&model.CallbackOutput{
                        Message:    lastEmptyMsg,
                        Config:     cbInput.Config,
                        TokenUsage: toModelCallbackUsage(lastEmptyMsg.ResponseMeta),
                    }, nil)
                }
                return
            }

            if chunkErr != nil {
                _ = sw.Send(nil, fmt.Errorf("failed to receive stream chunk from DeepSeek: %w", chunkErr))
                return
            }

            msg, found := resolveStreamResponse(chunk)
            if !found {
                continue
            }

            if lastEmptyMsg != nil {
                cMsg, cErr := schema.ConcatMessages([]*schema.Message{lastEmptyMsg, msg})
                if cErr != nil {
                    _ = sw.Send(nil, fmt.Errorf("failed to concatenate stream messages: %w", cErr))
                    return
                }

                msg = cMsg
            }

            if msg.Content == "" && len(msg.ToolCalls) == 0 {
                if _, ok := GetReasoningContent(msg); !ok {
                    lastEmptyMsg = msg
                    continue
                }
            }

            lastEmptyMsg = nil

            closed := sw.Send(&model.CallbackOutput{
                Message:    msg,
                Config:     cbInput.Config,
                TokenUsage: toModelCallbackUsage(msg.ResponseMeta),
            }, nil)

            if closed {
                return
            }
        }

    }()

    ctx, nsr := callbacks.OnEndWithStreamOutput(ctx, schema.StreamReaderWithConvert(sr,
        func(src *model.CallbackOutput) (callbacks.CallbackOutput, error) {
            return src, nil
        }))

    outStream = schema.StreamReaderWithConvert(nsr,
        func(src callbacks.CallbackOutput) (*schema.Message, error) {
            s := src.(*model.CallbackOutput)
            if s.Message == nil {
                return nil, schema.ErrNoValue
            }

            return s.Message, nil
        },
    )

    return outStream, nil
}

Stream 方法在for循环中从ChatCompletionStream流中读取响应的内容， 并发送至PIPE， 方法最后将PIPE封装为schema.StreamReader对象返回， 后续只需从schema.StreamReader循环读取模型响应流式回答内容, 如下所示：

func reportStream(sr *schema.StreamReader[*schema.Message]) {
    defer sr.Close()

    i := 0
    for {
        message, err := sr.Recv()
        if err == io.EOF {
            return
        }
        if err != nil {
            log.Fatalf("recv failed: %v", err)
        }
        log.Printf("message[%d]: %+v\n", i, message)
        i++
    }
}