Class: Vangrail::Engine

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/engine.rb

Overview

Runs ordered rails over text and reports one Result.

The rules are short enough to state in full:

  • Rails run in the order given. The first :blocked ends the pass.
  • A :modified result replaces the text for every rail after it, and the engine reports :modified unless something later blocks.
  • A rail that raises is not a rail that passed. on_error: :allow (the default) keeps going and marks the pass uncertain; :block stops.
  • A rail that ran and could not decide is the same fact arriving as a verdict rather than an exception: an endpoint that timed out, a budget spent before the answer, a model that would not hold the contract. on_uncertain: :allow (the default) reports the pass with certain? false; :block refuses instead.
  • An empty rail list returns :passed with certain false. Nothing ran.

Which of those two a deployment wants is a real choice and the published systems disagree. Meta's LlamaFirewall fails closed: its alignment scanner answers "treating as potentially compromised for safety" when evaluation errors. The default here is the other way, because a documentation desk that refuses to answer whenever its judge is unreachable is a desk that is down, and because certain? hands the caller the fact either way. A deployment where the cost of answering unchecked is higher than the cost of not answering sets on_uncertain: :block and gets the other policy.

Threading rewrites through later rails is the part worth being explicit about: a redaction rail that runs before a policy rail should have the policy rail judge the redacted text, not the original.

Constant Summary collapse

Screening = 
Vangrail::Screening
Triage = 
Vangrail::Triage

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input: [], context: [], output: [], on_error: :allow, on_uncertain: :allow, cache: true) ⇒ Engine

Returns a new instance of Engine.

Raises:

  • (ArgumentError) 


45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
 
# File 'lib/vangrail/engine.rb', line 45

def initialize(input: [], context: [], output: [], on_error: :allow,
               on_uncertain: :allow, cache: true)
  @input_rails = Array(input)
  @context_rails = Array(context)
  @output_rails = Array(output)
  @on_error = on_error.to_sym
  raise ArgumentError, 'on_error must be :allow or :block' unless %i[allow block].include?(@on_error)

  @on_uncertain = on_uncertain.to_sym
  unless %i[allow block].include?(@on_uncertain)
    raise ArgumentError, 'on_uncertain must be :allow or :block'
  end

  @cache = cache.is_a?(ResultCache) ? cache : (ResultCache.new if cache)
end

Instance Attribute Details

#cache ⇒ Object (readonly)

Returns the value of attribute cache.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def cache
  @cache
end

#context_rails ⇒ Object (readonly)

Returns the value of attribute context_rails.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def context_rails
  @context_rails
end

#input_rails ⇒ Object (readonly)

Returns the value of attribute input_rails.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def input_rails
  @input_rails
end

#on_error ⇒ Object (readonly)

Returns the value of attribute on_error.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def on_error
  @on_error
end

#on_uncertain ⇒ Object (readonly)

Returns the value of attribute on_uncertain.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def on_uncertain
  @on_uncertain
end

#output_rails ⇒ Object (readonly)

Returns the value of attribute output_rails.



43
44
45
 
# File 'lib/vangrail/engine.rb', line 43

def output_rails
  @output_rails
end

Instance Method Details

#assess ⇒ Object 



76
 
# File 'lib/vangrail/engine.rb', line 76

def assess(...) = Assessor.new(self).assess(...)

#check_context(text, **context) ⇒ Object

One retrieved document, before it goes anywhere near a prompt.



70
71
72
 
# File 'lib/vangrail/engine.rb', line 70

def check_context(text, **context)
  run(:context, context_rails, text, context)
end

#check_input(text, context = {}) ⇒ Object 



61
62
63
 
# File 'lib/vangrail/engine.rb', line 61

def check_input(text, context = {})
  run(:input, input_rails, text, context)
end

#check_output(text, user_input: nil, passages: nil, **context) ⇒ Object 



65
66
67
 
# File 'lib/vangrail/engine.rb', line 65

def check_output(text, user_input: nil, passages: nil, **context)
  run(:output, output_rails, text, context.merge(user_input: user_input, passages: passages))
end

#describe ⇒ Object 



115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/vangrail/engine.rb', line 115

def describe
  return 'no rails' if empty?

  parts = []
  parts << "input=#{rail_names(:input).join('+')}" unless input_rails.empty?
  parts << "context=#{rail_names(:context).join('+')}" unless context_rails.empty?
  parts << "output=#{rail_names(:output).join('+')}" unless output_rails.empty?
  parts << "on_error=#{on_error}"
  parts << 'offline' if offline?
  parts.join(' ')
end

#empty? ⇒ Boolean

Returns:

  • (Boolean) 


100
101
102
 
# File 'lib/vangrail/engine.rb', line 100

def empty?
  input_rails.empty? && context_rails.empty? && output_rails.empty?
end

#invoke(rail, text, ctx) ⇒ Object

Public so Assessor can run one rail without going through #run.



128
129
130
 
# File 'lib/vangrail/engine.rb', line 128

def invoke(rail, text, ctx)
  memoized(rail, text, ctx) { call_rail(rail, text, ctx) }
end

#offline? ⇒ Boolean

True when every configured rail decides without a network call, which is the only case where an unreachable endpoint cannot weaken the check.

Returns:

  • (Boolean) 


95
96
97
98
 
# File 'lib/vangrail/engine.rb', line 95

def offline?
  all = input_rails + context_rails + output_rails
  !all.empty? && all.all?(&:offline?)
end

#rail_names(side) ⇒ Object 



89
90
91
 
# File 'lib/vangrail/engine.rb', line 89

def rail_names(side)
  rails(side).map(&:name)
end

#rails(side) ⇒ Object 



80
81
82
83
84
85
86
87
 
# File 'lib/vangrail/engine.rb', line 80

def rails(side)
  case side.to_sym
  when :input then input_rails
  when :context then context_rails
  when :output then output_rails
  else raise ArgumentError, "unknown side: #{side}"
  end
end

#screen ⇒ Object 



74
 
# File 'lib/vangrail/engine.rb', line 74

def screen(...) = Screening.run(self, ...)

#to_h ⇒ Object 



104
105
106
107
108
109
110
111
112
113
 
# File 'lib/vangrail/engine.rb', line 104

def to_h
  {
    'input' => rail_names(:input),
    'context' => (rail_names(:context) unless context_rails.empty?),
    'output' => rail_names(:output),
    'on_error' => on_error.to_s,
    'offline' => offline?,
    'cache' => cache&.to_h,
  }.compact
end

#triage ⇒ Object 



78
 
# File 'lib/vangrail/engine.rb', line 78

def triage(...) = Assessor.new(self).triage(...)