Does this refer to just small talk, or does the author not care about the conversation he is having with these people? But why wouldn't he care, does he not care about the people, or does he think that he is above them?

I think the poet is writing about how the sounds of nature is what brings him peace. The buzzing of bees, crickets chirp, sounds of the water etc. He seems to find peace in the simple things that nature has to over, outside of man made noise.