It should be possible to find current menu state from java code. The way it is done for Swing, JMenuBar and JMenuItem and JMenu components, although invisible, reflect selected/expanded properties which are actually set in Mac OS X native menu. It does not matter whether this would be done the same way in FX or somehow differently as long as a test code is able to determine menu/submenu selection.
This is required for automation of single app menu bar testing. Rest could be done similar to how it is done in Jemmy v2 for Swing.