Persona 1 is horribly dated. It's incredibly oldschool and unless you like olden dungeon crawlers you're not going to like the game at all. I liked it a fair bit because I'm used to first person dungeon crawls, but I wouldn't really recommend it because the game becomes bullshit fairly quickly and if you're not a vet of SMT, you're going to get crushed easily. I still have bad memories of the Snow Queen Quest. The only reason you'd ever want to play it is if you want to see how it ties into P2, because some of the characters make cameos or become playable there. It's not necessary to play P2 though.
Persona 3 on the PSP is a mixed bag. It's not the definitive version of the game, but it has the ability to play as a female and have new social links. I'd suggest you'd get the FES version on the PS2/PSN. It doesn't have those problems you mentioned because of the PSP's memory limits. You'll be able to travel on foot, than on a cursor and not just see talking portraits. It also has the epilogue where you play as Aegis and wraps up the entire story nicely. (Which incidentally ties to the Persona Fighting game.) The problem with the console version though, is that the menus don't look as nice, longer load times, and none of the improvements of 4. So, no dizzy and no control over your teammates. Overall, the game is pretty good, but the plot really drags on in the middle with few important or interesting events. I only ever suggest P3P for people that played and loved FES and want a bit more.
Persona 2 is great, has awesome characters and has a fantastic storyline. It's only real weakness is that it's brutally easy. I'd suggest sticking to Hard mode (which is still too easy), but the game still doesn't get hard until the end. It's also surprisingly short, but I think that's a good thing. To avoid confusion, Innocent Sin is the first half of the story and Eternal Punishment being the conclusion. Did I mention you can battle Hitler and Nazis in this game?
Persona 4 is easily the best out of the bunch and the easiest to get into. Has a strong emotionally charged storyline, great characters, AMAZING VOICEACTING (I can't stress this enough, Troy Baker <3), and a pretty decent combat system. My only issue is that it's difficulty curve is wonky. It's hard at first, then gets significantly easier, then progressively harder. The Vita version is looking to be a definitive version unlike P3P, with all features intact, then better graphics, more story events, more voiceovers, new personas and social links. It's also portable and I love grinding on work breaks.
----
Izlude, I'll quit hating on Blizzard if they'd actually stop releasing the same games with no improvements on a 10+ year cycle. It's fucking ridiculous. I don't care how oldschool it is, the fact you wasted so much time to make something so average is appalling. It's sad that the last great game they made was Warcraft III because that was actually a vast improvement over II. As is such, if they're charging 60 bucks for an "old school" experience, I'd rather take it on something cheaper like Torchlight, instead of some overhyped bullshit. I don't see how such an acclaimed studio gets a free pass to make cheap, dated games with no originality whatsoever over and over. Even Last time I checked Skyward Sword didn't play like Zelda 1. Fuck Blizzard.
