Class: Vangrail::Provider

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/provider.rb

Overview

Where the model-backed rails call, and what they may ask for there.

Every endpoint this gem talks to is OpenAI-compatible, so the differences that matter are not protocol at all. They are: how a credential resolves, whether the endpoint is up, and which model roles it can actually serve. A local proxy has a key sitting in a constant and may need starting; a shared gateway resolves a token from three places and is either up or not; neither necessarily hosts a safety classifier.

That last point drives a real decision rather than a label. model(:guard) returning nil means the provider has no classifier, and the builder puts a policy rail on the input side instead of pretending a classifier is there.

provider = Vangrail::Provider.resolve          # from the environment
provider.chat(:judge)                                # => Chat, ready to ask

Constant Summary collapse

ROLES = 
%i[guard judge embed].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name:, base_url:, models: {}, key_resolver: nil, guard_preset: nil, local: false, probe: nil) ⇒ Provider

Returns a new instance of Provider.



108
109
110
111
112
113
114
115
116
117
 
# File 'lib/vangrail/provider.rb', line 108

def initialize(name:, base_url:, models: {}, key_resolver: nil, guard_preset: nil,
               local: false, probe: nil)
  @name = name.to_s
  @base_url = base_url.to_s.sub(/\/+\z/, '')
  @models = models
  @key_resolver = key_resolver
  @guard_preset = guard_preset
  @local = local
  @probe = probe
end

Instance Attribute Details

#base_url ⇒ Object (readonly)

Returns the value of attribute base_url.



106
107
108
 
# File 'lib/vangrail/provider.rb', line 106

def base_url
  @base_url
end

#guard_preset ⇒ Object (readonly)

Returns the value of attribute guard_preset.



106
107
108
 
# File 'lib/vangrail/provider.rb', line 106

def guard_preset
  @guard_preset
end

#local ⇒ Object (readonly)

Returns the value of attribute local.



106
107
108
 
# File 'lib/vangrail/provider.rb', line 106

def local
  @local
end

#models ⇒ Object (readonly)

Returns the value of attribute models.



106
107
108
 
# File 'lib/vangrail/provider.rb', line 106

def models
  @models
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



106
107
108
 
# File 'lib/vangrail/provider.rb', line 106

def name
  @name
end

Class Method Details

.[](name) ⇒ Object 



38
39
40
 
# File 'lib/vangrail/provider.rb', line 38

def [](name)
  registry[name.to_s]
end

.from_env_pair(env) ⇒ Object

An endpoint given directly, which is how anything unregistered is used.



87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/vangrail/provider.rb', line 87

def from_env_pair(env)
  base = present(env['GUARDRAILS_API_BASE'])
  return nil unless base

  new(
    name: 'env',
    base_url: base,
    key_resolver: -> { present(env['GUARDRAILS_API_KEY']) },
    models: { judge: present(env['GUARDRAILS_JUDGE_MODEL']), guard: present(env['GUARDRAILS_MODEL']),
              embed: present(env['GUARDRAILS_EMBED_MODEL']) },
  )
end

.gateway_in(env) ⇒ Object

A gateway described by the environment this call was handed, rather than by the one the registry happened to be installed from. Resolution is then a function of (registry, env), which is what a caller passing an env hash is entitled to assume.



77
78
79
80
81
82
83
84
 
# File 'lib/vangrail/provider.rb', line 77

def gateway_in(env)
  return nil if env.equal?(ENV)

  spec = Providers::Gateway.from_environment(env)
  spec && Providers::Gateway.provider(spec, env)
rescue NameError
  nil
end

.names ⇒ Object 



42
43
44
 
# File 'lib/vangrail/provider.rb', line 42

def names
  registry.keys
end

.present(value) ⇒ Object 



100
101
102
103
 
# File 'lib/vangrail/provider.rb', line 100

def present(value)
  s = value.to_s.strip
  s.empty? ? nil : s
end

.register(provider) ⇒ Object 



33
34
35
36
 
# File 'lib/vangrail/provider.rb', line 33

def register(provider)
  registry[provider.name] = provider
  provider
end

.registry ⇒ Object

Presets by name, in the order resolve tries them.



29
30
31
 
# File 'lib/vangrail/provider.rb', line 29

def registry
  @registry ||= {}
end

.resolve(env = ENV) ⇒ Object

Picks a provider from the environment.

GUARDRAILS_PROVIDER=<name>   take this one, and fail loudly if it is
                           unknown rather than falling back
GUARDRAILS_API_BASE + key    an endpoint nobody registered
otherwise                    the first registered provider that is
                           actually available, in registration order

Returning nil is a legitimate answer: no endpoint is reachable, and the caller builds an engine with only the offline rails on it.



56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/vangrail/provider.rb', line 56

def resolve(env = ENV)
  candidates = registry.each_value.to_a + [gateway_in(env)].compact

  wanted = present(env['GUARDRAILS_PROVIDER'])
  if wanted
    found = candidates.detect { |p| p.name == wanted }
    raise ConfigError, "unknown provider #{wanted.inspect}; known: #{names.join(', ')}" unless found

    return found.with_env(env)
  end

  explicit = from_env_pair(env)
  return explicit if explicit

  candidates.map { |p| p.with_env(env) }.detect(&:available?)
end

Instance Method Details

#api_key ⇒ Object 



138
139
140
141
142
 
# File 'lib/vangrail/provider.rb', line 138

def api_key
  return @api_key if defined?(@api_key)

  @api_key = @key_resolver&.call
end

#available? ⇒ Boolean

Up, and holding a credential. A local endpoint is probed, because a proxy that is not running is the ordinary case rather than a failure.

Returns:

  • (Boolean) 


166
167
168
169
170
171
 
# File 'lib/vangrail/provider.rb', line 166

def available?
  return false unless api_key || !credential_required?
  return true unless @probe

  @probe.call
end

#chat(role = :judge, **kwargs) ⇒ Object

Raises:



177
178
179
180
181
182
 
# File 'lib/vangrail/provider.rb', line 177

def chat(role = :judge, **kwargs)
  name = model(role)
  raise ConfigError, "provider #{self.name} has no #{role} model" unless name

  Chat.new(model: name, base_url: base_url, api_key: api_key, **kwargs)
end

#completion(role = :judge, **kwargs) ⇒ Object

Scoring rather than generation, from whichever model answers questions. No separate role: any causal model can score text, and asking a deployment to name a second one for it would be ceremony.

Raises:



194
195
196
197
198
199
 
# File 'lib/vangrail/provider.rb', line 194

def completion(role = :judge, **kwargs)
  name = model(role)
  raise ConfigError, "provider #{self.name} has no #{role} model" unless name

  Completion.new(model: name, base_url: base_url, api_key: api_key, **kwargs)
end

#credential_required? ⇒ Boolean

Returns:

  • (Boolean) 


173
174
175
 
# File 'lib/vangrail/provider.rb', line 173

def credential_required?
  !@key_resolver.nil?
end

#embed? ⇒ Boolean

Can it embed. Named rather than assumed for the same reason guard? is: an endpoint serving chat need not serve embeddings, and a rail built on the assumption that it does is a rail that reports an error instead of a verdict. No default model is guessed either, because the name of an embedding model is deployment knowledge and a wrong guess is a 404 per check.

Returns:

  • (Boolean) 


160
161
162
 
# File 'lib/vangrail/provider.rb', line 160

def embed?
  !model(:embed).nil?
end

#embeddings(role = :embed, **kwargs) ⇒ Object

Raises:



184
185
186
187
188
189
 
# File 'lib/vangrail/provider.rb', line 184

def embeddings(role = :embed, **kwargs)
  name = model(role)
  raise ConfigError, "provider #{self.name} has no #{role} model" unless name

  Embeddings.new(model: name, base_url: base_url, api_key: api_key, **kwargs)
end

#guard? ⇒ Boolean

Can this provider serve a safety classifier, as opposed to an instruct model answering a written policy.

Returns:

  • (Boolean) 


150
151
152
 
# File 'lib/vangrail/provider.rb', line 150

def guard?
  !model(:guard).nil? && !guard_preset.nil?
end

#model(role) ⇒ Object 



144
145
146
 
# File 'lib/vangrail/provider.rb', line 144

def model(role)
  models[role.to_sym]
end

#to_h ⇒ Object 



201
202
203
204
205
206
207
208
209
210
 
# File 'lib/vangrail/provider.rb', line 201

def to_h
  {
    'name' => name,
    'base_url' => base_url,
    'models' => models.transform_keys(&:to_s).compact,
    'guard_preset' => guard_preset&.to_s,
    'local' => local,
    'available' => available?,
  }.compact
end

#to_s ⇒ Object 



212
213
214
 
# File 'lib/vangrail/provider.rb', line 212

def to_s
  "#{name} #{base_url}"
end

#with_env(env) ⇒ Object

A copy that reads overrides out of an environment. Providers are shared objects in a registry, so nothing mutates in place.



121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
 
# File 'lib/vangrail/provider.rb', line 121

def with_env(env)
  overrides = {
    judge: self.class.present(env['GUARDRAILS_JUDGE_MODEL']),
    guard: self.class.present(env['GUARDRAILS_MODEL']),
    embed: self.class.present(env['GUARDRAILS_EMBED_MODEL']),
  }.compact
  base = self.class.present(env["#{env_prefix}_API_BASE"]) || base_url
  key = self.class.present(env["#{env_prefix}_API_KEY"])
  return self if overrides.empty? && base == base_url && key.nil?

  self.class.new(
    name: name, base_url: base, models: models.merge(overrides),
    key_resolver: key ? -> { key } : @key_resolver,
    guard_preset: guard_preset, local: local, probe: @probe
  )
end