Embedding Boolean circuits inside trained neural networks
Embedding boolean circuits in a neural network
A neural network neuron is just a weighted sum plus a bias passed through an activation, so if inputs are forced to 0 or 1 and the activation is "greater than zero," AND, OR, and NOT become single neurons. The harder part is wiring that circuit into an already-trained model: probes convert real-valued neuron outputs into boolean signals, and stims inject signed pushes along a neuron's output direction. The probe must fire in an earlier layer than the stim it triggers.
So you need an ingress and egress mapping: from the neural net into the boolean circuit, and then back into the neural net. Kind of like Neuralink but for LLMs.
- PaulHoule
I remember people talking about how we'd need neurosymbolic A.I. in the 1990s and imagining a large number of ways to do it but it all seemed pretty hypothetical because the neural networks weren't so good and now that neural representations of language are mature all the paths that you think would work are open.
- vatsachak
Okay but the difficulty here is getting the LLM to use the neural circuit effectively without training just wiping out the circuit. You'd need to have the circuit be a differentiable black box where gradients cannot affect the black box
- gnarlouse
Okay, so when do we get FPGAs in LLMs