Why has this (much) younger, "street rat" got to me? The more he didn't want a relationship, the more I pushed?

He played you because he wanted to use you. You describe him as a "street rat", is he semi-homeless or something? If you want a stable relationship it should always be with a stable person, not someone who needs you for anything. In my experience, anyone who could be described as a street rat is very passionate, very charming and manipulative, and also live their lives on the principle that they are prepared to lose or get rid of anything and everything in their lives if it is not benefiting them. It sounds like he worked very hard to try to string you along when it was convenient to him, but ultimately decided that he couldn't handle doing it for very long. People who are so used to loss are like that. They tend to throw people away out of fear of being thrown away first.