Class: Vangrail::Chat

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/chat.rb

Overview

One OpenAI-compatible chat call, shared by every model-backed rail.

Rails care about the text a model returned and how long it took. Everything else about the protocol lives here, so a rail is a prompt and a parser.

No endpoint is assumed. A Chat is built from a Provider, or from a base URL given outright; there is no vendor to fall back to, because a guardrail that quietly picks its own endpoint is one nobody can audit.

Defined Under Namespace

Classes: Answer

Constant Summary collapse

COMPLETIONS_PATH = 
'/chat/completions'

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model:, base_url: nil, api_key: nil, http: nil, max_tokens: 128, temperature: 0, extra: {}, open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT, read_timeout: 20) ⇒ Chat

Returns a new instance of Chat.



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/vangrail/chat.rb', line 22

def initialize(model:, base_url: nil, api_key: nil, http: nil, max_tokens: 128,
               temperature: 0, extra: {}, open_timeout: HTTP::DEFAULT_OPEN_TIMEOUT,
               read_timeout: 20)
  if http.nil? && base_url.to_s.strip.empty?
    raise ArgumentError,
          'a Chat needs a base_url or an http client'
  end

  @model = model
  @max_tokens = max_tokens
  @temperature = temperature
  @extra = extra
  @http = http || HTTP.new(
    base_url: base_url,
    api_key: api_key,
    open_timeout: open_timeout,
    read_timeout: read_timeout,
  )
end

Instance Attribute Details

#extra ⇒ Object (readonly)

Returns the value of attribute extra.



20
21
22
 
# File 'lib/vangrail/chat.rb', line 20

def extra
  @extra
end

#http ⇒ Object (readonly)

Returns the value of attribute http.



20
21
22
 
# File 'lib/vangrail/chat.rb', line 20

def http
  @http
end

#max_tokens ⇒ Object (readonly)

Returns the value of attribute max_tokens.



20
21
22
 
# File 'lib/vangrail/chat.rb', line 20

def max_tokens
  @max_tokens
end

#model ⇒ Object (readonly)

Returns the value of attribute model.



20
21
22
 
# File 'lib/vangrail/chat.rb', line 20

def model
  @model
end

#temperature ⇒ Object (readonly)

Returns the value of attribute temperature.



20
21
22
 
# File 'lib/vangrail/chat.rb', line 20

def temperature
  @temperature
end

Instance Method Details

#ask(messages = nil, conversation: nil, system: nil, max_tokens: nil) ⇒ Object

conversation: is the application path: the messages are whatever Conversation#messages will assemble, so a retrieved page cannot be spliced into the instruction by handing this method a raw array.

Raises:

  • (ArgumentError) 


53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/vangrail/chat.rb', line 53

def ask(messages = nil, conversation: nil, system: nil, max_tokens: nil)
  if conversation
    raise ArgumentError, 'pass conversation: or messages, not both' if messages

    messages = conversation.messages(system: system.to_s)
  end
  raise ArgumentError, 'ask needs messages or conversation:' if messages.nil?

  payload = {
    'model' => model,
    'messages' => normalize(messages),
    'temperature' => temperature,
    'max_tokens' => max_tokens || @max_tokens,
    'stream' => false,
  }.merge(extra)

  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  body = http.post_json(COMPLETIONS_PATH, payload)
  ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
  Answer.new(text: content_of(body), latency_ms: ms, raw: body)
end

#with(model:, max_tokens: nil, extra: {}) ⇒ Object

A copy pointed at a different model on the same endpoint and credentials.



43
44
45
46
47
48
 
# File 'lib/vangrail/chat.rb', line 43

def with(model:, max_tokens: nil, extra: {})
  self.class.new(
    model: model, http: http, temperature: temperature,
    max_tokens: max_tokens || @max_tokens, extra: extra
  )
end