Now is the very best time in all of history to have this worry. Why's that, you ask?
Well, there are more of us, for one thing. Know how all those editors got laid off? Well, luckily for you, many of them have turned to agenting: the skills are similar, and since there's no need to acquire, say, a license from the state to practice (strange that it's possible to sell books, but not houses, without one), that's where many editors have ended up. Yup. More potential agents for you.
Also, not to brag, but the average agent is a pretty cool human being. I know this because I've met lots of them. If they ran the world, I'd be perfectly pleased. The average agent is bright, kind, literary (one would hope), able to write well, able to pitch well, in touch with a number of great editors, great at editing manuscripts, amazing at teaching at conferences AND still manages to have an interesting social life involving all manner of hobbies.
All of this, to GK, equals rather likable. And likability in great supply.
Whatever you prefer--whatever you want along the geeky reader/MBA-type, aggressive/sweet, excitable/strictly professional spectrums--it's probably out there waiting for you.
And it just seems a law of the (you'll see my NorCal roots here) universe that the people you like are also the ones most likely to like you. It seems highly unlikely that someone who loves your work is going to be a terrible fit for you--perhaps not the best fit, and perhaps you won't hang out on purpose unless it's to discuss your work--but this agent, in order to fall in love with your work, had to get it. And, by extension, get you.
Do I think you should sign with any old agent because they offer you rep? No. Do I think you should say no to an agent because he/she lives on take-out and you (and your non-cookbook book) are all about the home cooked meals? Or because he/she likes cats and you like dogs? Or because you've never been to New York and don't like technology but he/she uses a Kindle and lives in Manhattan? No.
I think it's like the difference between a Hollywood love story and the more relaxed way most of us find our significant others.
There don't need to be ridiculous coincidences (though there often are) or signs. You just need to trust them, trust their vision, trust their communication style, trust their business sense, and trust that they're a good, solid choice. Liking them is also important. How do you feel about their client list? How do you feel about their proposed edits?
These things are far more important than whether you have eating, pet, and geographical preferences in common.
Realistically--as I've heard from many authors--there will likely be a moment when you just know.
Note that you don't have to be best friends with your agent. You don't have to go on coffee dates to chat and catch up; you don't even have to send holiday cards. You just have to be able to work with him/her and trust him/her to do his/her very best for your career.
The only problem I foresee is that it's difficult to find the newest of agents, since many works like The Jeff Herman Guide only come out yearly--and not all agents submit their information. So there are probably many more agents out there than it would appear. Check out the GLA blog (my friend Josh is featured today); many new agents are interviewed there quite often.
But there are also more resources (and for free!) available online than ever before.
So, yes. I understand this worry. But if you're going to be worried about this, of all things--now's the perfect time.