There's always a lot of luck and chance involved. I keep my eyes open to spot new opportunities. In this case, last year while researching http attacks, I discovered an information link on a major CDN. It looked promising, but I didn't have time to investigate it further because I was working on my HTTP desync research for Black Hat USA.
I looked back into it as soon as I was done with desync and saw that the mental model, I'd built for caching issues was way too simple. And where you've got complexity, you've got something!
Self-directed research is the only way research works. There's a massive list of things I would like to research, but I try and prioritize and use Black Hat submissions as a deadline for getting research done.