I don't think a Real Bout 3 or a Garou 2 is a good idea at this point. I mean what does the SNK community outside of Japan want to do? Bring new gamers in. We have an influx of new people due to SFIV, Blazblue, etc...and while maybe to some people, story and character development doesn't matter? It seems to matter to some people since its still present in games. That being said...Real Bout 1 is the only RB game that HAD a story, which was a continuation heavily dependent upon the story of Fatal Fury 1-3. RBS and RB2 have no story, and there's not much reason to make it "Real Bout 3" if its got a story. And most people wouldn't get why its called "Real Bout 3". it'd be a Street Fighter II situation where people didn't think there WAS a Street Fighter 1 lol.
Garou 2 would make sense, but at the same time...its been over ten years, and while the plot and setting behind the Fatal Fury series has a style that one could argue is more appealing to a western audience (American protagonists, American setting, etc...), I think Garou's heavy dependance upon the previous FF stories kind of kills it and makes it difficult for new comers to jump in. Not as many people played this series as they did with SFII, and the way the series is set up it'd be really hard to do a "SFIV" style game that can bring back the old and bring in new.
So my proposition? Completely reboot Fatal Fury. Start over from the beginning. Completely remake Fatal Fury 1 with 3D polygons on 2D plane with the line-shift system from RB2 so there's only two planes. Implement some Real Bout mechanics and maybe even a few from Garou, maybe something new (as long as it isn't botched) and there ya go. Include a few new fighters into the mix as well. If SNKP could pull all that off ontop of heavy marketing, I think they could have a heavy contender for SFIV/SFxT. They would just need to make sure the characters feel COMPLETE this time and do thorough balance tests with real players. Balance patches of course need to be implemented in some way. And for a finishing touch, GGPO for the console version. I think that entire package right there could re-ignite SNK and Fatal Fury in the western world and really make them a viable contender in today's market (Which all SNK fans SHOULD want btw).
If not this, then I'd have to say do a new Samurai Shodown in the same style as SFIV, with bringing in mostly returning guys from the core series and a few new faces, but make it so the story is easy for old fans and new ones alike to jump into. And of course, balanced gameplay with GGPO on the console version. And 3D polygons. I love 2D games, don't get me wrong, but Seth Killian himself said it: "3D sells". Sales figures back him up here I'm afraid.