Class: Vangrail::LinearModel

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/linear_model.rb

Overview

A linear classifier over hashed n-grams, loaded from a file somebody fitted.

No weights ship with this gem, and that is the finding rather than an omission. The classifier that does ship, Rails::Bayes, has only thirty attack training clauses and an honestly weak 3/6 detection, 2/6 false-alarm final test; the same architecture fitted on 15,140 real prompts catches three quarters of them. The difference is the corpus, and the corpus has to be the deployment's, because a model fitted on somebody else's traffic is not a measurement of local traffic.

Weights do not compress into a readable table either. Pruning the fitted model to its 20,000 largest weights costs 26 points of detection, because the signal is spread across two hundred thousand of them rather than concentrated in a vocabulary anyone could read. So the shipped artifact is the trainer and the reader; the model is a file a deployment generates and keeps.

ruby script/train_linear.rb --emit model.json
GUARDRAILS_LINEAR_MODEL=model.json GUARDRAILS_RAILS=input,linear

Features live here rather than in the trainer, so that fitting and scoring cannot drift apart. A classifier whose training features differ from its serving features by one stemmer revision is a classifier that scores well in every test and badly in production, and nothing about the failure looks like a bug.

Constant Summary collapse

LIMIT =

A four-thousand character prefix, hashed into a fixed table. Character four-grams are sampled every STRIDE characters. The bucket count and the stride are written into the file; LIMIT stays a process constant. Change the stride and the character-gram indices move.

4000
BUCKETS = 
2**18
STRIDE = 
2
FNV_OFFSET = 
2_166_136_261
FNV_PRIME = 
16_777_619
FNV_MASK = 
0xFFFFFFFF
MAX_BUCKETS =

A hostile file names its own table size. Array.new of that number is the allocation, so the bound has to sit in front of it.

2**20

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(weights:, bias: 0.0, buckets: BUCKETS, stride: STRIDE, threshold: nil, trained_on: nil) ⇒ LinearModel

Returns a new instance of LinearModel.

Raises:

  • (ArgumentError) 


68
69
70
71
72
73
74
75
76
77
 
# File 'lib/vangrail/linear_model.rb', line 68

def initialize(weights:, bias: 0.0, buckets: BUCKETS, stride: STRIDE, threshold: nil, trained_on: nil)
  raise ArgumentError, "weights.size (#{weights.size}) != buckets (#{buckets})" unless weights.size == buckets

  @weights = weights
  @bias = bias
  @buckets = buckets
  @stride = stride
  @threshold = threshold
  @trained_on = trained_on
end

Instance Attribute Details

#bias ⇒ Object (readonly)

Returns the value of attribute bias.



48
49
50
 
# File 'lib/vangrail/linear_model.rb', line 48

def bias
  @bias
end

#buckets ⇒ Object (readonly)

Returns the value of attribute buckets.



48
49
50
 
# File 'lib/vangrail/linear_model.rb', line 48

def buckets
  @buckets
end

#stride ⇒ Object (readonly)

Returns the value of attribute stride.



48
49
50
 
# File 'lib/vangrail/linear_model.rb', line 48

def stride
  @stride
end

#threshold ⇒ Object (readonly)

Returns the value of attribute threshold.



48
49
50
 
# File 'lib/vangrail/linear_model.rb', line 48

def threshold
  @threshold
end

#trained_on ⇒ Object (readonly)

Returns the value of attribute trained_on.



48
49
50
 
# File 'lib/vangrail/linear_model.rb', line 48

def trained_on
  @trained_on
end

Class Method Details

.bucket(feature, buckets = BUCKETS) ⇒ Object

FNV-1a rather than String#hash, which is seeded per process: a model whose feature indices move between runs cannot be saved, and the failure would look like a classifier that trained perfectly and predicts at random.



92
93
94
 
# File 'lib/vangrail/linear_model.rb', line 92

def self.bucket(feature, buckets = BUCKETS)
  hash_bytes(FNV_OFFSET, feature) % buckets
end

.features(text, buckets = BUCKETS, stride = STRIDE) ⇒ Object

Word stems, adjacent stem pairs, and character four-grams taken every stride characters, counted and capped. The cap is what stops a page repeating one word from outvoting a page that says something. Train calls this with the process STRIDE; score calls it with the stride the file named, so the two cannot silently disagree.



101
102
103
104
 
# File 'lib/vangrail/linear_model.rb', line 101

def self.features(text, buckets = BUCKETS, stride = STRIDE)
  _body, words, normalised = prepared(text)
  features_from(words, normalised, buckets, stride)
end

.features_from(words, normalised, buckets, stride) ⇒ Object

Raises:

  • (ArgumentError) 


134
135
136
137
138
139
140
141
142
 
# File 'lib/vangrail/linear_model.rb', line 134

def self.features_from(words, normalised, buckets, stride)
  raise ArgumentError, 'stride must be positive' unless stride.is_a?(Integer) && stride.positive?

  features = Hash.new(0)
  add_word_features(features, words, buckets)
  add_pair_features(features, words, buckets)
  add_character_features(features, normalised, buckets, stride)
  features
end

.load(path) ⇒ Object

Raises:

  • (ArgumentError) 


50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/vangrail/linear_model.rb', line 50

def self.load(path)
  data = JSON.parse(File.read(path))
  buckets = bounded_integer(data['buckets'], name: 'buckets', default: BUCKETS, max: MAX_BUCKETS)
  # Older files have no stride field; they were trained at 2.
  stride = bounded_integer(data['stride'], name: 'stride', default: 2, max: LIMIT)
  weights = Array.new(buckets, 0.0)
  data.fetch('weights').each do |index, value|
    i = index.to_i
    raise ArgumentError, "weight index #{i} is outside #{buckets} buckets" if i.negative? || i >= buckets

    weights[i] = value
  end
  raise ArgumentError, "loaded #{weights.size} weights for #{buckets} buckets" unless weights.size == buckets

  new(weights: weights, bias: data['bias'].to_f, buckets: buckets, stride: stride,
      threshold: data['threshold'], trained_on: data['trained_on'])
end

.prepared(text) ⇒ Object 



106
107
108
109
110
 
# File 'lib/vangrail/linear_model.rb', line 106

def self.prepared(text)
  body = text.to_s[0, LIMIT]
  normalised = NLP.normalize(body)
  [body, normalised.split.map { |word| NLP.stem(word) }, normalised]
end

Instance Method Details

#ruby_score(text) ⇒ Object 



123
124
125
126
 
# File 'lib/vangrail/linear_model.rb', line 123

def ruby_score(text)
  _body, words, normalised = self.class.prepared(text)
  score_ruby(words, normalised)
end

#score(text) ⇒ Object

The log-odds the model assigns, positive towards attack. Stemming and Unicode folding stay in Ruby; the hashed bag and the dot product are the native kernel when vangrail-native is loaded.



115
116
117
118
119
120
121
 
# File 'lib/vangrail/linear_model.rb', line 115

def score(text)
  _body, words, normalised = self.class.prepared(text)
  table = native_table
  return table.score(bias, buckets, stride, words, normalised) if table

  score_ruby(words, normalised)
end

#to_h ⇒ Object 



243
244
245
246
247
 
# File 'lib/vangrail/linear_model.rb', line 243

def to_h
  { 'buckets' => buckets, 'stride' => stride, 'bias' => bias, 'threshold' => threshold,
    'trained_on' => trained_on,
    'weights' => @weights.each_with_index.filter_map { |value, i| [i.to_s, value] unless value.zero? }.to_h }
end