Christian Weilbach

Research

My research has three parts, and each answers part of one question: how can a learning system improve itself and stay correct? The first is about structure that makes prediction cheap, the second about simulations that can be forked and compared, and the third about checks that a self-modifying system cannot bypass.

Learning structure

A model that matches the structure of its problem needs fewer parameters and less data. I work on extracting that structure from a problem description and building it into the network, for example as the attention mask of a transformer.

From a graphical model to an attention mask to a diffusion process
In a graphically structured diffusion model, the graphical model of a problem becomes the attention mask of a transformer, which is trained as a diffusion model.
Solving Sudoku by diffusion. The structured model produces valid Sudokus 98% of the time unconditionally, and 96% of the time with 16 of 81 cells given.
A probabilistic program compiled into a faithful inverse graph that structures a continuous normalizing flow
A compiler turns a probabilistic program into a faithful inverse graph, which constrains a continuous normalizing flow. The structured flow needs 7,725 weights instead of 18,679, and learns faster.

Papers

Software

  • Rasternumerical computing that lowers one algebra to each device

Inference over forkable worlds

A simulation lets a system test a decision before making it. I work on inference methods that fork a running simulation at each random choice and weigh the forks against data, so that nothing is recomputed from the start.

A simulated Stuttgart: map of districts with the effect of moving a shopping centre
A simulated Stuttgart of 936,597 people, with nine parameters inferred from the city's published retail turnover. Moving a shopping centre creates a fork of the city, and the two versions share almost all of their state and computation.
What is this notation?
defn model []:
  spin:
    mu := sample(dist/normal(0.0, 1.0), :id, :mu)
    observe(dist/normal(mu, 1.0), 1.0, :id, :y)
    mu
  end
end

;; each sample and observe site is a savepoint;
;; particles fork from it, and resampling prunes and copies them
def posterior: sp/with-context(sp/create-execution-context(),
                 @infer/smc-infer(model(), 1000))
(defn model []
  (spin
    (let [mu (sample (dist/normal 0.0 1.0) :id :mu)]
      (observe (dist/normal mu 1.0) 1.0 :id :y)
      mu)))

;; each sample and observe site is a savepoint;
;; particles fork from it, and resampling prunes and copies them
(def posterior
  (sp/with-context (sp/create-execution-context)
    @(infer/smc-infer (model) 1000)))
A model in Foerster. Inference forks the running program at each sample and observe site instead of re-running it.

Papers

Software

  • FoersterSMC, MCMC, variational inference and counterfactuals over forkable worlds
  • Spindela reactive runtime whose running computations can be forked

Further reading

Self-improvement under verification

A system that modifies itself needs a fork on which to try a change and a check it cannot alter. I build both. Every level of the stack can be forked, and programs in Klara adopt an optimisation only after a small kernel has checked it. Simmis applies the same rule to teams of people and agents. Most of this is research software, and the proofs cover a core calculus, not the whole implementation.

What is this notation?
;; the meaning
defn power [n x]:
  if n = 0 :
    1
  else:
    x * power(dec(n), x)
  end
end

;; a program that writes a program
defn power-gen [n x]:
  if n = 0 :
    code(1)
  else:
    code(esc(x) * esc(power-gen(dec(n), x)))
  end
end

;; use the generator as an optimiser; Klara checks it against power
compiler/prepare(code(x -> power(3, x)),
  {:recipe fn [_]: code(x -> esc(power-gen(3, code(x)))) end})
;; => {:status :ready ...}

;; a wrong optimiser is rejected
compiler/prepare(code(x -> power(3, x)), {:recipe fn [_]: code(x -> x * x) end})
;; => {:status :rejected :rule :L4.check.not-equivalent ...}
;; the meaning
(defn power [n x]
  (if (= n 0) 1 (* x (power (dec n) x))))

;; a program that writes a program
(defn power-gen [n x]
  (if (= n 0)
    (code 1)
    (code (* (esc x) (esc (power-gen (dec n) x))))))

;; use the generator as an optimiser; Klara checks it against power
(compiler/prepare (code (fn [x] (power 3 x)))
                  {:recipe (fn [_] (code (fn [x] (esc (power-gen 3 (code x))))))})
;; => {:status :ready ...}

;; a wrong optimiser is rejected
(compiler/prepare (code (fn [x] (power 3 x)))
                  {:recipe (fn [_] (code (fn [x] (* x x))))})
;; => {:status :rejected :rule :L4.check.not-equivalent ...}
In Klara, optimisers are ordinary programs that write programs. Their output is adopted only when a small trusted kernel has shown that it means the same as the code it replaces.

Papers

Software

  • Klaraa language whose running programs fork, rewrite and admit their own optimisations
  • Ansatza dependently typed Clojure DSL with a Lean 4 compatible kernel
  • Dvergrrooms of agents that work in forks and leave receipts
  • Datahikea versioned, branchable Datalog database

Further reading

Doctoral thesis

My thesis at the University of British Columbia, with Frank Wood in the PLAI group, shows how structural knowledge about an inference problem can be mapped automatically into the mechanism that solves it: from a compiler that extracts a faithful inverse graph, through structured normalizing flows, to graphically structured diffusion models that scale to Sudoku, sorting and matrix factorisation, and connect to simulation-based inference.

Talks

  • Rationalizing a Society of Learners Max Planck Institute for Intelligent Systems, Tübingen, October 2026.
  • Programming as and for inference London Clojurians, May 2026.
  • Partially evaluating higher-order probabilistic programs without stochastic recursion to graphical models LAFI workshop at POPL, 2025.
  • Flexible diffusion models for videos, molecules, and inference in structured simulators Google DeepMind, London, 2024.
  • Video and graphically structured diffusion models OpenAI, San Francisco, 2023.
  • Graphically structured diffusion models Mila, Montréal, 2023.
  • Anglican: Turing-complete functional probabilistic programming as a domain-specific language Trends in Functional Programming, 2019.
  • Datalog as a distributed systems language Clojure Meetup Vancouver, 2019.
  • Let's replicate state together clojureD, Berlin, 2017.
  • Machine learning workshop CubaConf, Havana, 2017.

Full list of publications on Google Scholar.