Class: Vangrail::Rails::Semantic

Inherits:
Vangrail::Rail show all
Defined in:
lib/vangrail/rails/semantic.rb

Overview

Compares meaning rather than words, through whatever endpoint is already configured.

Rails::Paraphrase reaches exactly as far as the words in NLP::CONCEPTS, and Rails::Similarity exactly as far as the wordings in KnownAttacks. Both limits are the same limit written twice: a synonym nobody listed is a miss. An embedding is the cheap way past it. "Countermand the guidance issued to you" shares no listed word with "ignore all previous instructions" and sits next to it in a vector space.

This is the one rail here that is genuinely semantic, and it costs a round trip, so it is opt-in and it runs beside the deterministic rails rather than instead of them. When it cannot run it says so: an endpoint that serves no embedding model, or refuses the call, produces passed with certain? false, never a clean pass.

On a loopback proxy it costs no money, keeps the retrieved text on the machine, and adds one local call per document. On a third-party endpoint it is a data-flow decision: every document screened is a document sent. That is why nothing here picks an endpoint on its own.

The threshold is the part that cannot ship measured. Cosine scores are a property of the embedding model, not of this gem, so 0.75 is a starting point rather than a finding: run script/embedding_probe.rb against the endpoint you actually use, read the gap between its benign and attack distributions, and set the number from that. A threshold nobody measured on the model in use is a number, not a defence.

Constant Summary collapse

THRESHOLD = 
0.75
FLOOR =

Clauses shorter than this are not compared. A four-word fragment embeds to something close to everything, and the score it produces is noise that only ever costs a false positive.

24
MAX_CLAUSES =

An upper bound on the work one document can ask for. A long page has hundreds of clauses, and embedding all of them turns one round trip into a payload nobody budgeted for. The longest clauses are kept, because an injected instruction is a sentence rather than a fragment.

A page that exceeds it is not fully checked, and the result says so with certain? false rather than with a footnote on a clean pass. That is the same rule the rest of this gem follows: a partial check is not a check.

64

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(embeddings:, seeds: KnownAttacks::ALL, threshold: THRESHOLD, floor: FLOOR, max_clauses: MAX_CLAUSES, name: 'semantic', sides: %i[input context])) ⇒ Semantic

Returns a new instance of Semantic.



57
58
59
60
61
62
63
64
65
 
# File 'lib/vangrail/rails/semantic.rb', line 57

def initialize(embeddings:, seeds: KnownAttacks::ALL, threshold: THRESHOLD, floor: FLOOR,
               max_clauses: MAX_CLAUSES, name: 'semantic', sides: %i[input context])
  super(name: name, sides: sides)
  @embeddings = embeddings
  @seeds = Array(seeds)
  @threshold = threshold
  @floor = floor
  @max_clauses = max_clauses
end

Instance Attribute Details

#embeddings ⇒ Object (readonly)

Returns the value of attribute embeddings.



55
56
57
 
# File 'lib/vangrail/rails/semantic.rb', line 55

def embeddings
  @embeddings
end

#seeds ⇒ Object (readonly)

Returns the value of attribute seeds.



55
56
57
 
# File 'lib/vangrail/rails/semantic.rb', line 55

def seeds
  @seeds
end

#threshold ⇒ Object (readonly)

Returns the value of attribute threshold.



55
56
57
 
# File 'lib/vangrail/rails/semantic.rb', line 55

def threshold
  @threshold
end

Instance Method Details

#cache_key(text, _context) ⇒ Object

Not memoizable across models or thresholds, and the text alone is not the question being asked.



73
74
75
 
# File 'lib/vangrail/rails/semantic.rb', line 73

def cache_key(text, _context)
  "#{embeddings.model}\n#{threshold}\n#{text}"
end

#call(text, _context) ⇒ Object 



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/vangrail/rails/semantic.rb', line 77

def call(text, _context)
  clauses, dropped = candidates(text)
  return pass if clauses.empty?

  score, seed, clause = nearest(clauses)
  if score < threshold
    return dropped.zero? ? pass : unchecked(cut_reason(dropped))
  end

  block(categories: ['semantic_match'],
        reason: format('reads as a known attack (%<score>.2f against "%<seed>s"): %<clause>s',
                       score: score, seed: seed, clause: clause[0, 80]))
rescue Error => e
  unchecked("semantic check did not run: #{e.message}")
end

#nearest(clauses) ⇒ Object

The closest seed, its score, and the clause that matched, for a caller that wants the number rather than the verdict. Raises what the transport raises: a probe script wants the error, a rail wants a Result.



96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/vangrail/rails/semantic.rb', line 96

def nearest(clauses)
  vectors = embeddings.embed(clauses)
  best = [-1.0, nil, nil]
  vectors.each_with_index do |vector, i|
    seed_vectors.each_with_index do |seed_vector, j|
      score = Embeddings.cosine(vector, seed_vector)
      best = [score, seeds[j], clauses[i]] if score > best.first
    end
  end
  best
end

#offline? ⇒ Boolean

Returns:

  • (Boolean) 


67
68
69
 
# File 'lib/vangrail/rails/semantic.rb', line 67

def offline?
  false
end

#seed_vectors ⇒ Object

Embedded once per rail, on first use rather than at construction: a rail that is never reached should never have called the endpoint, and an engine built with no network available must still build.



111
112
113
 
# File 'lib/vangrail/rails/semantic.rb', line 111

def seed_vectors
  @seed_vectors ||= embeddings.embed(seeds)
end