How come I can't hear dialogue while playing a game?

My console originally went to HDMI out but since I use a AV receiver for all of my consoles I had switched it to DIGITAL OUT. I tried again with a game and I heard the other missing dialogues better but other pieces of dialogue became quieter. What should I do to correct this audio problem? I use two external speakers with the AV receiver if that helps to get a better answer. Also, there are three checkmark options for digital out; AAC, Dolby 5.1, and I believe DTS 5.1. There is also another option to choose bistreams; linear PCM, dolby bistream, and DTS bistream. Which of those should I use along with the first part of my question? Thanks in advance.