Module: Vangrail::Watermark

Defined in:
lib/vangrail/watermark.rb

Overview

A machine-readable mark saying text was generated, carried in the text itself.

Article 50(2) of the AI Act requires providers of generative systems to mark their output in a machine-readable format, detectable as artificially generated, by solutions that are effective, interoperable, robust and reliable as far as technically feasible. Recital 133 names watermarks and cryptographic provenance among the techniques meant. The second subparagraph of Article 50(4) adds a narrower duty on the deployer, for text published to inform the public, met by a sentence a reader can see. This is the first duty: a property of the bytes.

None of the sampler-side schemes can do this job for us. The green-list tilt (doi:10.48550/arXiv.2301.10226), tournament sampling (doi:10.1038/s41586-024-08025-4), and the cryptographic construction (doi:10.48550/arXiv.2306.09194) all live inside token selection, and an application holding a key for somebody else's endpoint does not get to touch it. Under bring-your-own-key it is not even one endpoint. What the application does own is the text after it arrives, so the mark goes there: deterministic, identical for every model, whichever provider answered.

Asking the model to sign its own output is the other non-answer. It fails on exactly the cases the obligation is about: a small model, a long context, a terse-output instruction, a provider that trims trailing lines.

The format

Eleven bytes per marked segment, encoded one byte per variation selector and appended to the segment's last character:

0-1  MAGIC, 0xA1 0x50, public and fixed
2    VERSION, currently 1
3-10 TAG, HMAC-SHA256 truncated to eight bytes, or eight zero bytes

Variation selectors carry a byte each: 0x00 to 0x0F as U+FE00 to U+FE0F, and 0x10 to 0xFF as U+E0100 to U+E01EF. They render as nothing after a base character that has no variant form, they survive a copy through a browser, an editor, and a mail client that keeps Unicode, and Rails::Obfuscation leaves a run of them alone on the output side.

Two levels of reading, which is the point of splitting magic from tag:

anybody      finds MAGIC and VERSION and knows the text is generated. No
           key, no agreement with us, eleven lines of code against the
           published layout. That is what interoperable has to mean, and
           it is the same argument the publicly-detectable schemes make
           on the sampler side (doi:10.48550/arXiv.2310.18491).
the issuer   recomputes the HMAC and knows the text is theirs, and that
           the mark was not lifted off another answer and pasted on.

The tag covers the segment it is attached to, canonicalised: marks removed, whitespace runs collapsed, ends trimmed. So a mail client that rewraps the paragraph still verifies, quoting one paragraph out of six still verifies on that paragraph, and moving a mark onto different words does not.

What it does not survive

Retyping, a transcription, an ASCII-only pipeline, or any tool that strips format characters. A distributional mark degrades under paraphrase and can still be measured (doi:10.48550/arXiv.2306.04634); this one disappears, leaving no partial signal. Nothing published survives a determined rewrite either (doi:10.48550/arXiv.2303.13408), which is why a visible sentence in the application is the other half and this is not provenance on its own.

Code is never marked. A variation selector inside a shell command is a command that fails, or worse, one that runs differently, so fenced and indented blocks come back byte for byte.

Defined Under Namespace

Classes: Report

Constant Summary collapse

MAGIC = 
[0xA1, 0x50].freeze
VERSION = 
1
TAG_BYTES = 
8
UNSIGNED = 
[0].freeze * TAG_BYTES
LENGTH = 
MAGIC.length + 1 + TAG_BYTES
DOMAIN =

The domain string keeps this HMAC from ever matching one computed for another purpose with the same key.

'vangrail/watermark/v1'
LOW =

One byte per selector. Sixteen in the BMP and the rest in the supplement, because a byte needs 256 values.

0xFE00
HIGH = 
0xE0100
SELECTOR = 
/[\u{FE00}-\u{FE0F}\u{E0100}-\u{E01EF}]+/
FENCE =

A fence, or four spaces at the start of a line. Both are code, and code is copied into a terminal.

/^[ \t]*(?:```|~~~)/
INDENTED = 
/^(?: {4}|\t)/

Class Method Summary collapse

Class Method Details

.authentic_segment?(segment, key: nil, issuer: nil) ⇒ Boolean

Returns:

  • (Boolean) 


234
235
236
237
238
239
 
# File 'lib/vangrail/watermark.rb', line 234

def authentic_segment?(segment, key: nil, issuer: nil)
  return false if key.nil? || key.to_s.empty?

  wanted = tag(strip(segment), key: key, issuer: issuer)
  marks(segment).any? { |m| m[:version] == VERSION && m[:tag] == wanted }
end

.canonical(text) ⇒ Object

What the tag is computed over. Rewrapping a paragraph must not break a verification, and neither must a trailing space a renderer added.



230
231
232
 
# File 'lib/vangrail/watermark.rb', line 230

def canonical(text)
  strip(text).gsub(/\s+/, ' ').strip
end

.decode(run) ⇒ Object

A run of selectors back to the byte strings in it, split on MAGIC so a segment carrying two marks reports two.



194
195
196
197
198
199
200
 
# File 'lib/vangrail/watermark.rb', line 194

def decode(run)
  bytes = run.each_char.map do |c|
    cp = c.ord
    cp < HIGH ? cp - LOW : (cp - HIGH) + 0x10
  end
  split_on_magic(bytes)
end

.encode(bytes) ⇒ Object

Bytes to selectors.



188
189
190
 
# File 'lib/vangrail/watermark.rb', line 188

def encode(bytes)
  bytes.map { |b| [b < 0x10 ? LOW + b : HIGH + (b - 0x10)].pack('U') }.join
end

.map_segments(text) ⇒ Object 



250
251
252
253
254
255
256
257
258
259
260
261
 
# File 'lib/vangrail/watermark.rb', line 250

def map_segments(text)
  in_fence = false
  segments(text).map do |segment|
    fences = segment.scan(FENCE).length
    was_open = in_fence
    in_fence = !in_fence if fences.odd?
    next segment if was_open || fences.positive?
    next segment if segment.match?(/\A\s*\z/) || segment.match?(INDENTED)

    yield segment
  end.join
end

.mark(text, key: nil, issuer: nil) ⇒ Object

The marked text. Idempotent: a segment that already carries a valid mark for this key is left as it is, so a rail can run twice without stacking selectors.



100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/vangrail/watermark.rb', line 100

def mark(text, key: nil, issuer: nil)
  map_segments(text.to_s) do |segment|
    stripped = strip(segment)
    next segment if stripped.strip.empty?
    next segment if authentic_segment?(segment, key: key, issuer: issuer)

    selectors = encode(payload(stripped, key: key, issuer: issuer))
    # Before the trailing newline, not after it: a selector run needs a base
    # character in front of it or a renderer draws a dotted box for it.
    stripped.sub(/(\s*)\z/) { "#{selectors}#{::Regexp.last_match(1)}" }
  end
end

.marked?(text) ⇒ Boolean

Is this text marked as generated. No key, because that is the question the obligation is about: anybody holding the text can ask it.

Returns:

  • (Boolean) 


123
124
125
 
# File 'lib/vangrail/watermark.rb', line 123

def marked?(text)
  marks(text).any?
end

.marks(text) ⇒ Object

Every mark found, as { version:, tag: }, in the order they appear.



128
129
130
131
132
133
134
135
 
# File 'lib/vangrail/watermark.rb', line 128

def marks(text)
  text.to_s.scan(SELECTOR).flat_map { |run| decode(run) }
      .filter_map do |bytes|
    next unless bytes[0, MAGIC.length] == MAGIC

    { version: bytes[MAGIC.length], tag: bytes[(MAGIC.length + 1)..] }
  end
end

.payload(canonical_source, key: nil, issuer: nil) ⇒ Object 



217
218
219
 
# File 'lib/vangrail/watermark.rb', line 217

def payload(canonical_source, key: nil, issuer: nil)
  MAGIC + [VERSION] + tag(canonical_source, key: key, issuer: issuer)
end

.segments(text) ⇒ Object

Paragraphs, with fenced and indented code kept whole and left alone. The separators stay in the list so a round trip through map_segments returns the original spacing.



246
247
248
 
# File 'lib/vangrail/watermark.rb', line 246

def segments(text)
  text.split(/(\n[ \t]*\n)/)
end

.split_on_magic(bytes) ⇒ Object 



202
203
204
205
206
207
208
209
210
211
212
213
214
215
 
# File 'lib/vangrail/watermark.rb', line 202

def split_on_magic(bytes)
  out = []
  index = 0
  while index < bytes.length
    if bytes[index, MAGIC.length] == MAGIC && bytes.length - index >= LENGTH
      out << bytes[index, LENGTH]
      index += LENGTH
    else
      out << [bytes[index]]
      index += 1
    end
  end
  out
end

.strip(text) ⇒ Object

Every mark removed, and nothing else touched.



114
115
116
117
118
119
 
# File 'lib/vangrail/watermark.rb', line 114

def strip(text)
  text.to_s.gsub(SELECTOR) do |run|
    kept = decode(run).reject { |bytes| bytes[0, MAGIC.length] == MAGIC }
    kept.map { |bytes| encode(bytes) }.join
  end
end

.tag(text, key: nil, issuer: nil) ⇒ Object 



221
222
223
224
225
226
 
# File 'lib/vangrail/watermark.rb', line 221

def tag(text, key: nil, issuer: nil)
  return UNSIGNED.dup if key.nil? || key.to_s.empty?

  message = "#{DOMAIN}\n#{issuer}\n#{canonical(text)}"
  OpenSSL::HMAC.digest('SHA256', key.to_s, message).bytes[0, TAG_BYTES]
end

.verify(text, key: nil, issuer: nil) ⇒ Object

Per segment: marked, and if a key was given, ours.

report = Vangrail::Watermark.verify(answer, key: ENV['ASK_WATERMARK_KEY'])
report.marked?      # something in here says generated
report.authentic?   # and at least one mark is this issuer's
report.coverage     # share of segments carrying our mark


143
144
145
146
147
148
149
150
151
 
# File 'lib/vangrail/watermark.rb', line 143

def verify(text, key: nil, issuer: nil)
  segments = segments(text.to_s).reject { |segment| strip(segment).strip.empty? }
  rows = segments.map do |segment|
    { text: strip(segment),
      marked: marked?(segment),
      authentic: !key.nil? && authentic_segment?(segment, key: key, issuer: issuer) }
  end
  Report.new(rows)
end