I saw a similar project about 6 years ago at Chaos Communication Congress in Berlin - so I can't take credit for the solar-cell-as-sensor idea. (Unfortunately, I don't remember who made that - some university I think. It was a bit strange architechture as well - realtime patched Windows PC with Matlab/Simulink etc.)

The algorithm is not that sophisticated - no modeling involved. 2 cascaded control loops - the outer one has wheel speed as input and ball position (solar cell target voltage) as output, the inner one has solar cell voltage as input and motor pwm as output (plus a feedback from motor speed to pwm - so the output from the inner controller is closer to acceleration than speed). Whole controller structure: Image

The spinning motion in the video comes from the fact that the feedback from motor speed was tuned at higher speeds and less load (table tennis ball). So for the steel ball I have quite a big control deviation (distance of the red an green line) on the inner controller. The i part of the outer controller "learns" this away, and after the spinning direction changes, the deviation is in the opposite direction: Image