Ah, so it's a microswitch joystick and not leaf. Anyway you found the problem.
Weird hack they did. The wires of the microswitches are hooked up to where the optical units were....they could just have as well connected it directly to the harness.
Anyway, the good thing is that the original joystick connector now still is there.
The board is quite useless without the optical units, they are apparently very hard to find today.