> mkv and avi are just containers.
> MKV files usually have MPEG-4 video with AC3 or AAC audio.
> AVI can also have MPEG-4 video with AC3 / AAC audio tracks.
> afaik, MAME records uncompressed video and audio, and stores them in a AVI file.
This. MKV is just a container format. It's not like it would magically make MAME emit well-compressed videos that look any good.
And open-source or not, AVI is still the de facto standard for video interchange when doing video editing and such.