One of my best friends as a kid, I met at an ice-breaking activity in summer camp and became instant Real Friends with. So, it does occasionally happen like that...which may possibly be why your bafflers expect it to. :)

But usually, I find I have to agonize about how to transition someone from Sandbox Friend to Outside-the-Sandbox friend, especially since historically it's often involved me being the one to make the explicit overtures.