u need to check the output setting, is it bitstream or pcm.
if choose pcm, ps3 will do the decode, if choose bitstream, the amp will decode.
however, your av amp dosenot support, it's better to choose 5.1 but not dtshd, even u choose, ps3 will downsample or send pcm signal to av amp, that's mean it's still 5.1, but not truehd or dtshd.