Simply put, cognition is computation, computation operates over symbols, symbols begin with inputs to the brain and end with outputs from the brain, and so it is in the brain alone that cognition takes place, and it is with the brain alone that cognitive science need concern itself.
I think the brain is constantly looking for ways to optimize its conscious and unconscious cognitive processing, which would explain things like cognitive offloading without disproving this idea. (also seen in the idea of direct/indirect perception that we talked about in class)