Nobody else replied so you're stuck with my less-than-expert advice :P
You don't mention which title you are having trouble with, but for example, 4-way joystick arcade games (Pacman, and etc) don't know what to do when they receive multiple simultaneous inputs, such as which comes from an 8-way joystick pushed into the diagonals. The bizarre behavior you are experiencing may be that.
When configuring the joystick, you'll be pushing it cleanly up, down, right left, but while playing you'll also be using diagonals too, even if not intentional, which is sending multiple inputs to the game. This would explain the difference between correct behavior during configuration and bizarre behavior in-game.
Even worse, since you didn't specify what your "classic arcade joystick" was, it's also possible that your joystick is analogue, which is even worse!
This is a big part of why people that play MAME either have multiple joysticks (4-way, 8-way, and analogue), or at least a switchable 4-way/8-way joystick.
Now get out out there, and win this one for the home team!