Filter gives me the impression of inclusion... so if I filter by fruits, I expect to see apples, oranges, and bananas. Instead, this is more like filter out fruits... remove all the fruits, and you're left with the rest. Filter in/out are both viable. One means to include everything that matches a condition, and the other is to exclude everything that does not match a condition. And I don't think we can have just one.

The x mark is also sometimes used for this purpose (most notably on election ballot papers, e.g. in the United Kingdom), but otherwise usually indicates "no", incorrectness, or failure.