but I would not go while Mrs. Smith was there, and with no other companion than Mr. Willoughby.”
In the Regency era, it was common for people to visit private homes and estates. There was a "presumption of access to private houses" (Clarke, "A fine house richly furnished: Pemberley and the visiting of country houses") In this discussion, Elinor is chiding Marianne for breaking one of the cardinal rules of visiting a great house: visiting while the family who owned said house was at home. "If the family were away, [a] housekeeper would feel free to show the house to visitors" but "access [could be] refused because the family were at home" (Clarke). By entering Allenham while Mrs. Smith was at home, Marianne has intruded. The fact that she was also alone with Willoughby, a man who is not a relative of hers, with no supervision from others could cause people to question Marianne's virtue.
(Image: "Gold and enamel mourning ring with hair of the deceased under glass, c. 1855." from The Victor Mourning Blog)