Predict

There were many challenge problems that I encountered in my school years, but few of them are as memorable as the pendulum trial.

When I was a young child, I was taken to a high stairwell. There was a ball hanging from a long string several floors from above, held in place slightly to the side by an obstacle. “Predict how long it will take for the ball to swing back and forth, five times, once I remove this. You have a stopwatch, measurement tape, a ball, string and pen and paper.”

As went without saying, I wasn’t allowed to remove the obstacle myself and simply test. The intention was obviously that I should build a similar setup myself with the tools I had. But, as I quickly realized, the string I was given was much too short for this. I puzzled.

I did build a pendulum out of my tools, though, as that felt like the obvious thing to do. And I did time how long it took for it to swing back and forth. And I did realize that this is not necessarily the right answer to what I was asked, since my pendulum was much smaller.

So I spent a long time sitting around, thinking that the problem is impossible, and pretending that trying to stretch the string so that it would be long enough wasn’t an obvious dead end. But given enough time to get frustrated and start to do something, I did eventually test out several different-sized pendulums. And given enough time, I did forget what I had measured earlier, and also noticed this and started to write things down on paper.

I ultimately arrived at the idea of extrapolation, and had the numerical skills required to arrive at a good guess. I announced that I’m done and gave my best estimate for the time. I was then, as usual, offered a wide variety of bets – “3 : 1 that it will take more than a minute, up or down?” – to which I promptly gave my responses. Then came showtime, and I was so nervous, after which I was so excited to see getting it right.

Now that I’m older, I appreciate that I wasn’t the only one who succeeded that day. My success at predicting the pendulum meant that the educational experts had succeeded at predicting a child studying a pendulum.

Actually, that obfuscates the causality: it would be more appropriate to say that their success caused mine.

The educational experts, where I’m from, cared about children learning. As such, they went to great degrees to measure and predict whether children would in fact learn from particular activities.

The pendulum trial was seen as one of the key developmental milestones of early childhood science. It was crafted to allow children to succeed in discovering a deep, latent regularity in their surrounding world at as early an age as possible. The experts cared that children would in fact be able to solve it, so that the children would achieve the feeling of success, building their self-confidence in doing science and world-modeling. “Mom, mom! I did a scientific experiment today! I’m now a scientist too!” is the ideal outcome of the activity.

The predictive task is rather tame as psychological prediction goes. There is just one child in an environment that is completely controllable, in a domain that’s completely understood. I’ve looked at the statistical methods they use, and really with just a simple model of 11 parameters – some directly measurable like age, some which are latent variables like perseverance, aggregated from the data collected throughout the children’s education – one can get it right a supermajority of the time.

The real predictors are of course vastly more complicated, but I haven’t dug into what people actually use; all I know is they are still clearly more accurate and way more robust to statistical outliers. They have to be, for otherwise they’d lose out both gradually and all at once on the prediction markets, the ultimate mechanism for aggregating predictions and keeping the predictors accountable.

I don’t know what specific probability the market put on me succeeding before I entered the room, but I know it was more than 19 : 1. Any less than that and the environment would have had more subtle hints. Any less than that even with the hints and I wouldn’t have been let into the room at all, but would instead have had to wait, grow and learn more before the trial.

Typically, and especially at a later age, the threshold is much lower, since the optimal number of “oops that was too hard and he failed” isn’t zero – otherwise you’d just go faster and learn more, obviously. Besides, older kids are more resilient to failures, and failures are quite surprisingly often more teaching than successes.

There are indeed several failures I remember to this day. One of them was in those hypothesis-forming-games. I’ve forgotten the details of the flavour-text, but it was about how the consumption of various vegetables changes based on their prices and what the profit-maximizing prices are. I thought this was one of the easier ones, since you could sample a lot of new data points at the near-zero menu costs as long as you didn’t hike the prices too low or high.

I remembered seeing supply-demand curves in the economic books, and so knew that polynomials and rational functions are the right functional forms to go for. I Bayes-updated to fit the parameters, but it worked worse than I expected. I thought it was just noise, so spent some of the in-game money to get more data points on the regions where I did poorly. No help. I fit exponentials and radicals and logarithms and everything, but in my heart I didn’t really believe they would work much better, and indeed they didn’t. Eventually time was up.

Afterwards I was told that the true data generating process involved a step function around round, integer-valued prices. I got frustrated, and was about to protest that this was ridiculous and against the spirit of hypothesis-forming-games. I was silenced upon being informed that such an effect is also present in real life prices.

Much later afterwards I learned that this was the modal outcome the markets had predicted for me. They thought I would likely fail and that I would be frustrated about it, but it was thought to be the type of frustrating failure that would be valuable for personal growth. I don’t disagree.

A major reason the hypothesis-forming-games are so widely used in our education is, I’ve understood, that they’re among the most accessible ways we have found for training science skills and teaching children what doing science is like. Designing and running intricate psychological experiments involving multiple people, for example, requires actually quite a lot of time and effort and skill and care, and is thus not something we can do many times with our children as part of their education. Playing hypothesis-forming-games whose flavour-texts are about the interactions between aliens in turn is something that can be done at scale, providing a glimpse to what it’s like to be a psychologist.

Sometimes they beat you over the head with it. This was in the time when I hadn’t yet connected the dots, and was about to play another one of those games. Unsuspecting, I opened the flavour-text, and was taken by hilarious surprise when it was about children attempting to predict how long it will take for a pendulum to swing back and forth.