In short, Smith believed that slavery is not as profitable as free labor and that forced servitude takes away economic agents’ motivation to succeed. It takes away their ability to better their own condition. Instead, they are motivated to make life easier for themselves in the short term, which transfers costs that would normally be borne by the worker to the master.
This is closely related to current employment models. Many jobs simply pay by the hour, and don't promote or give raises out with no motivation for workers that do better.