Walled ghettos are a well-known example of physical segregation.90 Jewish people in Europe were made to live in separate, walled areas, as were Arab and European traders in China.
Keeping people behind walls and trapping them in and environment where it is impossible for them to grow is unfair. They should be able to integrate with other people so that there will be fair opportunities. Excluding a group doesn't help the situations they are in.