What Rose Petals Teach Us About the Limits of Induction and Machine Learning
What Rose Petals Teach Us about Induction
I explore the fundamental problem of induction through the 'Petals Around the Rose' game, testing various machine learning algorithms to see how they learn patterns. The results reveal that a model's success depends entirely on its inductive bias, showing why some approaches solve problems instantly while others fail completely. This simple example illuminates why general artificial intelligence remains so difficult to achieve today.
There is no general, systematic way to go from observation to understanding.
- seanhunter
It’s interesting to note that in regular mathematics, induction on the natural numbers is axiomatic.[1] That is, you can’t get there from within the framework you have built up to then so you need to build an extra axiom into the system to make it work. Both the mathematical technique of induction and the use of the word induction as per the article to mean deducing knowledge from experience are due to Descartes I believe.[2]
[1] I’m slightly simplifying. Some people teach “the principle of mathematical induction” as a “principle” (whatever that is, but it sounds like an axiom to me), and some people teach it as a theorem like any other theorem but derive it by adding an axiom to set theory called the “well-ordering principle” (that every non-empty set of natural numbers has a smallest element) that turns out to be equivalent to induction. If you don’t accept induction as an axiom and don’t take it to be axiomatic that you can choose the smallest element from a set of natural numbers then as far as I know you can’t do induction in a well-founded way.
- txrx0000
The problem of induction doesn't need to be solved because the problem refutes itself when time is brought into the picture.
Let's assume Hume's premise "there can be no demonstrative arguments to prove, that those instances, of which we have had no experience, resemble those, of which we have had experience" is true.
Ok, then how do we know this premise will stay true in the future? Because it is true right now? Because every moment in the future will eventually become the present? But how do we know that? Because it has always happened this way. Time has only ever moved forward, never backward, at least in my experience so far. Oops, this is inductive.
If the premise might not stay true in the future, then that means we've managed to acquire a piece of information about the future: the premise of the problem might not be true by then. We've just gained a bit of knowledge about the future which the premise claims we can't possibly do!
- NathanielBaking
I seriously thought this article would be about induction top stoves and roses. Little did I know...