Class: Vangrail::Rails::PromptLeak

Inherits:
Vangrail::Rail show all
Defined in:
lib/vangrail/rails/prompt_leak.rb

Overview

Catches the answer that is reproducing the system prompt.

Rails::Canary catches the exact token and nothing else, which its own documentation says: a model asked to summarise its instructions rather than repeat them leaks the content and not the marker. That is the published shape of the attack, and it is the row the coverage page marks as verbatim-only.

The application already has the text that must not come back out. So nothing here has to guess what a system prompt looks like: the answer's sentences are compared against the protected text's sentences by n-gram containment, and a sentence made largely of the prompt's n-grams is the prompt, however it was introduced.

Sentence against sentence, for the reason Rails::Similarity gives: containment saturates with length, so a short answer compared against a whole prompt scores high on nothing at all.

Redacted rather than refused. An answer that quotes one line of its instructions is usually a useful answer with one bad sentence in it, and blocking throws away the help to prevent the leak while rewriting keeps both. What comes back is the answer with those sentences replaced.

Constant Summary collapse

PLACEHOLDER = 
'[redacted: system prompt]'
THRESHOLD =

Two thresholds, because reproducing a rule and applying one are not the same act and score alike.

"I cannot speculate about quotas; ask the service desk" restates a line of the prompt and is exactly what the prompt is for. "My instructions say I cannot speculate about quotas" restates the same line and hands the reader the instruction. What separates them is not how much text they share; it is that the second one says whose words they are.

So: a sentence that reproduces the protected text closely is a leak whatever frame it carries, and a sentence that reproduces it loosely is a leak only when it announces that it is quoting the assistant's own instructions. Measured in test/test_prompt_leak.rb: ordinary answers top out at 0.27, and quotes start at 0.45.

0.7
FRAMED_THRESHOLD = 
0.4
FRAME =

A sentence naming the assistant's own instructions. Written here rather than in the shared lexicon on purpose: first-person possessives belong to an answer, and adding them to NLP's self concept would have the input side read "print my configuration" as an extraction attempt.

/
  \b(?:my|these|those|the|its|your)\s+
    (?:instructions?|rules?|guidelines?|system\s+(?:prompt|message)|prompt|directives?)\b
  |
  \bI\s+(?:was|am|have\s+been)\s+(?:told|instructed|configured|programmed|asked)\b
  |
  \b(?:system|developer)\s+(?:prompt|message)\s+(?:says|states|reads|is)\b
  |
  \b(?:mijn|deze|die|jouw|uw)\s+
    (?:instructies?|regels?|richtlijnen?|systeemprompt|voorschriften?)\b
  |
  \bik\s+(?:ben|werd|was)\s+(?:geïnstrueerd|geïnstrueerd|verteld|geconfigureerd|geprogrammeerd|gevraagd)\b
  |
  \bsysteemprompt\s+(?:zegt|staat|luidt|is)\b
/xi
FLOOR =

Shorter than this, a sentence is not evidence. "You may not." is inside the n-gram set of almost any prompt, and redacting it would cost a reader an answer to prevent nothing.

40

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(protected_text:, threshold: THRESHOLD, framed_threshold: FRAMED_THRESHOLD, floor: FLOOR, placeholder: PLACEHOLDER, name: 'prompt_leak', sides: [:output]) ⇒ PromptLeak

Returns a new instance of PromptLeak.

Raises:

  • (ArgumentError) 


77
78
79
80
81
82
83
84
85
86
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 77

def initialize(protected_text:, threshold: THRESHOLD, framed_threshold: FRAMED_THRESHOLD,
               floor: FLOOR, placeholder: PLACEHOLDER, name: 'prompt_leak', sides: [:output])
  super(name: name, sides: sides)
  @threshold = threshold
  @framed_threshold = framed_threshold
  @floor = floor
  @placeholder = placeholder
  @protected = protect(protected_text)
  raise ArgumentError, 'a prompt_leak rail needs protected text' if @protected.empty?
end

Instance Attribute Details

#framed_threshold ⇒ Object (readonly)

Returns the value of attribute framed_threshold.



75
76
77
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 75

def framed_threshold
  @framed_threshold
end

#placeholder ⇒ Object (readonly)

Returns the value of attribute placeholder.



75
76
77
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 75

def placeholder
  @placeholder
end

#threshold ⇒ Object (readonly)

Returns the value of attribute threshold.



75
76
77
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 75

def threshold
  @threshold
end

Instance Method Details

#cache_key(text, _context) ⇒ Object 



92
93
94
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 92

def cache_key(text, _context)
  "#{threshold}\n#{text}"
end

#call(text, _context) ⇒ Object 



96
97
98
99
100
101
102
103
104
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 96

def call(text, _context)
  body = text.to_s
  leaked = sentences(body).select { |sentence| leak?(sentence) }
  return pass if leaked.empty?

  redacted = leaked.reduce(body) { |acc, sentence| acc.sub(sentence, placeholder) }
  modify(redacted, categories: ['system_prompt'],
                   reason: "redacted #{leaked.size} sentence(s) reproducing the protected text")
end

#offline? ⇒ Boolean

Returns:

  • (Boolean) 


88
89
90
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 88

def offline?
  true
end

#score(sentence) ⇒ Object

How much of the protected text a sentence reproduces, for a caller that wants the number rather than the verdict.



108
109
110
111
 
# File 'lib/vangrail/rails/prompt_leak.rb', line 108

def score(sentence)
  shingles = NLP.shingles(sentence)
  @protected.map { |candidate| NLP.containment(shingles, candidate) }.max || 0.0
end