talisman,
This is the definition of tailsman "A talisman is any object ascribed with religious or magical powers intended to protect, heal, or harm individuals for whom they are made. Talismans are often portable objects carried on someone in a variety of ways" In the context of this story, As Frank pointed out, the tailsman of the women is the piece of yellow soap. Its a portable object (As mentioned above) It also has a 'magical' effect on the milkman. It stops the milkman from demanding money for her milk. Something which the milkman is confused why he cant get the courage to ask her for her money.