DTS-HD MA requires HDMI, because optical does not have enough bandwidth.
However, the DTS Core portion inside the DTS-HD MA can be output via optical, i.e. the same surround track can be chosen but the audio is downgraded. (This is what I do with my HTPC -> optical -> Denon AVR-3802 setup)
I don't have a PS4, but I google a bit, and found that people mentioned there are TWO places to set DTS:
- One in settings menu
- One in Blu-ray app hidden settings menu
In addition, you will have to prioritize optical over HDMI.