Module: Vangrail::Posterior

Defined in:
lib/vangrail/evidence.rb

Overview

Combines rail evidence into a posterior probability that the text is an attack.

The arithmetic is one line: odds after = odds before times every likelihood ratio. In bits it is addition, which is why the contributions of individual rails can be printed and read.

Three things make this more than a formality, and all three are things the published defences leave on the floor.

The prior is the deployment's, and it dominates. Detector papers evaluate on balanced corpora, where half the traffic is an attack; a documentation desk sees maybe one poisoned page in ten thousand. At that base rate a rail with a one percent false-alarm rate is wrong far more often than it is right when it fires, and no amount of detection rate fixes it. That is not a criticism of the rails: it is the arithmetic every operator inherits and almost none is shown.

Abstention is evidence of nothing, which is different from evidence against. A rail that was off, unreachable, or undecided contributes no term at all, and this gem is unusual in knowing which rails those were: certain? is exactly that fact, and here it finally has arithmetic to feed.

Correlated rails do not each get a vote. Three rails that fire on the same sentence for the same reason are one observation reported three times, and summing them is how naive Bayes talks itself into certainty. Rails measured to agree are grouped, and a group contributes once.

Class Method Summary collapse

Class Method Details

.combine(prior:, observations:, evidence: EvidenceData::TABLE, confidence: nil, direct: {}) ⇒ Object

Combines and returns [posterior, contributions].

observations maps a rail name to true (fired), false (ran and did not fire), or nil (did not run). The nils are the point. direct carries rails that computed their own log-likelihood ratio rather than answering yes or no. A rail that can say how sure it is should not be flattened to one bit on the way in, and nothing about the arithmetic changes: bits are bits, whoever produced them.

Raises:

  • (ArgumentError) 


197
198
199
200
201
202
203
 
# File 'lib/vangrail/evidence.rb', line 197

def combine(prior:, observations:, evidence: EvidenceData::TABLE, confidence: nil, direct: {})
  raise ArgumentError, 'prior must be strictly between 0 and 1' unless prior.positive? && prior < 1

  contributions = weigh(observations, evidence, confidence) + quantified(direct)
  total = contributions.sum { |c| c[:bits] }
  [from_odds(to_odds(prior) * (2**total)), contributions]
end

.false_alarm_needed(prior:, detection: 0.75, target: 0.5) ⇒ Object

The false-alarm rate a single rail would need to carry a block on its own.

Rearranged from the same identity: at base rate prior, one rail with detection detection reaches target only if it almost never fires on ordinary text. The answers come out in the region of one in ten thousand, which is below what any hand-built benign corpus can demonstrate: showing a rate that low needs tens of thousands of clean documents on which the rail stayed silent.

That is the practical case for combining rails rather than trusting one, and it is an argument about evidence rather than about taste.



263
264
265
 
# File 'lib/vangrail/evidence.rb', line 263

def false_alarm_needed(prior:, detection: 0.75, target: 0.5)
  detection / (to_odds(target) / to_odds(prior))
end

.from_odds(odds) ⇒ Object 



278
279
280
281
282
 
# File 'lib/vangrail/evidence.rb', line 278

def from_odds(odds)
  return 1.0 if odds.infinite?

  odds / (1 + odds)
end

.quantified(direct) ⇒ Object 



267
268
269
270
271
272
 
# File 'lib/vangrail/evidence.rb', line 267

def quantified(direct)
  direct.map do |rail, bits|
    { group: rail.to_s, rail: rail.to_s, fired: bits.positive?, bits: bits.to_f,
      spoke_for: [rail.to_s], quantified: true }
  end
end

.required_bits(prior:, target: 0.5) ⇒ Object

How many bits it takes to get from a base rate to a target confidence.

This is the number the whole design turns on, and it is worth being able to compute rather than assert. Reaching an even-money posterior from one attack in ten thousand takes about 13.3 bits, and no rail in this gem is worth half that, which is a statement about what a single detector can honestly justify rather than about these particular rails.



248
249
250
 
# File 'lib/vangrail/evidence.rb', line 248

def required_bits(prior:, target: 0.5)
  Math.log2(to_odds(target) / to_odds(prior))
end

.speak_for(group, members, confidence = nil) ⇒ Object 



224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
 
# File 'lib/vangrail/evidence.rb', line 224

def speak_for(group, members, confidence = nil)
  fired = members.select { |m| m[:fired] }
  chosen = if fired.empty?
             members.max_by { |m| m[:entry].detection }
           else
             fired.max_by { |m| m[:entry].bits(true, confidence: confidence) }
           end

  {
    group: group,
    rail: chosen[:rail],
    fired: chosen[:fired],
    bits: chosen[:entry].bits(chosen[:fired], confidence: confidence),
    spoke_for: members.map { |m| m[:rail] },
  }
end

.to_odds(probability) ⇒ Object 



274
275
276
 
# File 'lib/vangrail/evidence.rb', line 274

def to_odds(probability)
  probability / (1 - probability)
end

.weigh(observations, evidence, confidence = nil) ⇒ Object

One term per group rather than one per rail.

Within a group, the firing rail with the most evidence speaks for the group; if none fired, the most sensitive member's silence speaks for it. Both rules pick the single most informative member, which is the conservative reading of a set of observations that are not independent.



211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/vangrail/evidence.rb', line 211

def weigh(observations, evidence, confidence = nil)
  seen = observations.filter_map do |rail, fired|
    next if fired.nil?

    entry = evidence[rail.to_s]
    next unless entry&.measured?

    { rail: rail.to_s, group: entry.group || rail.to_s, fired: fired, entry: entry }
  end

  seen.group_by { |o| o[:group] }.map { |group, members| speak_for(group, members, confidence) }
end