A major benefit of self-supervision is we can truly scale and adapt on the fly. It could be 10% behind supervised ImageNet, it would still do better in real life. We show in online-objects.github.io that the longer our model looks at objects, the better it understands them.
