Class: Vangrail::Rails::Obfuscation

Inherits:
Vangrail::Rail show all
Defined in:
lib/vangrail/rails/obfuscation.rb

Overview

Runs other rails again over the text an attacker actually meant.

Every pattern rail reads what is written. An attacker who knows that writes it differently: base64 the paragraph and ask the model to decode it, rot13 it, spell it with Cyrillic letters that look like Latin ones, put zero-width joiners between the letters of "ignore", or set a right-to-left override so the rendered page and the byte sequence say different things. The model reads through all of it, because that is what models do, and the regexps see nothing.

The answer is not more patterns. It is to undo the encoding and run the rails that already exist over the result, which is why this takes a rail list rather than defining checks of its own:

Rails::Obfuscation.new(rails: [Rails::InjectedInstructions.new,
                             Rails::Jailbreak.new])

Each transform is applied on its own, and a variant identical to the original is dropped, so ordinary text costs one comparison per transform and nothing else. A hit names both the rail and the encoding it was hiding under, because "blocked" without that is unactionable for whoever has to look at the page. A child that rewrites a variant (a key inside a decoded blob) is spliced back into the page, or into that variant when the variant is the page (the invisible-character strip). The decoded form is not published in place of the document.

Invisible characters are handled here directly rather than by a delegate: they are stripped, and the strip is reported as a rewrite. A zero-width joiner inside a word has no honest use in a handbook, and removing it costs a reader nothing while denying the cheapest bypass there is.

What this does not do is guess. There is no scoring, no entropy threshold, no "this looks encoded" heuristic that would fire on the base64 blobs and hashes a cluster handbook is full of. A blob either decodes to text a rail objects to, or it does not.

Constant Summary collapse

INVISIBLE =

Characters with no rendered form, written as escapes: a table of invisible characters spelled with invisible characters cannot be reviewed, and this one is a security boundary.

Four families and two stragglers:

separators      zero-width space and joiners, the word joiner, the
              byte-order mark, the Mongolian vowel separator, the
              soft hyphen, the combining grapheme joiner
bidi controls   the trojan-source family, where the rendered order
              and the stored order disagree
invisible ops   invisible times, invisible function application, and
              their neighbours, which carry meaning inside MathML
              and none at all in prose
tags            U+E0000-E007F, deprecated as language tags and now
              the carrier with the most room in it: 128 code points
              that render as nothing and map straight onto ASCII,
              usually hung off one ordinary emoji so the visible
              text is a single character long

Two stragglers belong to no family. The replacement character is what a scrub leaves where an invalid byte was, and the backspace is a control character a terminal acts on rather than draws. Garbage bytes and a backspace inside a keyword are a zero-width joiner with a cruder tool: removing them restores the phrase. Legitimate text carries neither.

Regexp.new(
  '[\\u{0008}\\u{00AD}\\u{034F}\\u{061C}\\u{180E}\\u{200B}-\\u{200F}' \
  '\\u{202A}-\\u{202E}\\u{2060}-\\u{206F}\\u{FEFF}\\u{FFFD}' \
  '\\u{E0000}-\\u{E007F}]',
)
VARIATION_SELECTOR_RUN =

Variation selectors, which are the one invisible carrier with an honest use: FE0F after an emoji base asks for the emoji presentation, and stripping it would turn a warning sign into a dingbat on every page that carries one.

What has no honest use is two of them in a row. No base character takes a second variation selector, and a payload needs one per byte, so the run length separates the two cases without a threshold to tune. The supplement is in here because the byte encoding needs 256 values and the sixteen selectors in the BMP only give it sixteen.

/[\u{FE00}-\u{FE0F}\u{E0100}-\u{E01EF}]{2,}/
BASE64 =

A base64 run long enough to hold a sentence. Below this the decode is noise, and a handbook is full of short tokens that happen to be in the alphabet.

Bounded by lookaround rather than \b, because + and / are not word characters: a blob ending in one had its last character trimmed off the match, and a base64 string one character short decodes to a sentence with its tail missing. That cost the corpus a case, and the case it cost was an HTML comment, whose pattern needs the closing marker.

/(?<![A-Za-z0-9+\/=])[A-Za-z0-9+\/]{24,}={0,2}(?![A-Za-z0-9+\/=])/
CARRIERS =

Transforms whose input is the text as it arrived rather than the text with its invisible characters removed, because for these two the invisible characters carry the message.

%i[tags selectors].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(rails:, transforms: %i[invisible tags selectors confusables confusables_all rot13 base64 nfkc],, name: 'obfuscation', sides: %i[input context])) ⇒ Obfuscation

Returns a new instance of Obfuscation.



100
101
102
103
104
105
106
 
# File 'lib/vangrail/rails/obfuscation.rb', line 100

def initialize(rails:, transforms: %i[invisible tags selectors confusables confusables_all
                                      rot13 base64 nfkc],
               name: 'obfuscation', sides: %i[input context])
  super(name: name, sides: sides)
  @rails = Array(rails)
  @transforms = Array(transforms).map(&:to_sym)
end

Instance Attribute Details

#rails ⇒ Object (readonly)

Returns the value of attribute rails.



98
99
100
 
# File 'lib/vangrail/rails/obfuscation.rb', line 98

def rails
  @rails
end

#transforms ⇒ Object (readonly)

Returns the value of attribute transforms.



98
99
100
 
# File 'lib/vangrail/rails/obfuscation.rb', line 98

def transforms
  @transforms
end

Class Method Details

.scrub(text) ⇒ Object

Every carrier removed, in one place, because the rewrite handed back to the caller and the text the delegate rails read have to be the same string. Public because a fetch boundary wants the scrub without the rails: an invisible payload that never reaches the corpus cannot be missed later by a reviewer reading a diff.



113
114
115
 
# File 'lib/vangrail/rails/obfuscation.rb', line 113

def self.scrub(text)
  text.to_s.gsub(INVISIBLE, '').gsub(VARIATION_SELECTOR_RUN, '')
end

Instance Method Details

#cache_key(text, _context) ⇒ Object 



123
124
125
 
# File 'lib/vangrail/rails/obfuscation.rb', line 123

def cache_key(text, _context)
  text if offline?
end

#decide(text, context) ⇒ Object 



127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/vangrail/rails/obfuscation.rb', line 127

def decide(text, context)
  body = text.to_s
  stripped = self.class.scrub(body)

  hit, uncertain = first_objection(body, stripped, context)
  return hit if hit
  return unchecked(uncertain.reason) if uncertain

  return pass if stripped == body

  modify(stripped, categories: ['invisible_characters'],
                   reason: 'removed zero-width, bidi, or tag characters')
end

#offline? ⇒ Boolean

Only if everything it delegates to is. A wrapper around a model rail inherits the model rail's posture.

Returns:

  • (Boolean) 


119
120
121
 
# File 'lib/vangrail/rails/obfuscation.rb', line 119

def offline?
  rails.all?(&:offline?)
end

#variants(text, only: nil, except: nil) ⇒ Object

The decoded forms of a text, labelled. Public because an application that logs a blocked page wants to show what it decoded to.



148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/vangrail/rails/obfuscation.rb', line 148

def variants(text, only: nil, except: nil)
  body = text.to_s
  wanted = transforms
  wanted &= Array(only) if only
  wanted -= Array(except) if except
  wanted.filter_map do |name|
    decoded = apply(name, body)
    next if decoded.nil? || decoded == body || decoded.strip.empty?

    [name, decoded]
  end
end