Meaning that 99% of the people use it are using it "wrong" because they're not using it for HTTP authentication and don't send a WWW-Authenticate header field with their 401 response?

Hmm. That's a tough one. On the one hand, the spec does say they must send it.

Initial opinion

But on the other hand, one could argue that that requirement only applies if using 401 for HTTP authentication. And that saying it's wrong to do so (as they claim at https://stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses/14713094#14713094 and https://hyp.is/JA45zHotEeybDdM_In4frQ/stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses) is having a too strict/narrow/literal interpretation.

HTTP is meant to be used widely in many very different uses and contexts, most of which do not use this very specific HTTP authentication scheme; my opinion is that they shouldn't be denied from using it, just because they don't have anything useful WWW-Authenticate header field. (Or (which is also fine with me), just put something "emptyish" in the field, like "Unused". Unless that would trigger a Basic auth modal in the browser, in which case we shouldn't, for practical reasons.)

Why shouldn't we be able to repurpose this same status code for uses that are still authentication, but just not HTTP authentication per se?

Is it really wrong to repurpose this useful status code for other contexts, like cookie-based app-defined authentication systems?

I say that it's okay to repurpose/reuse 401 for any authentication system (that uses HTTP as a part of it, even though not using HTTP's own authentication system), as long as we try to maintain the same semantic as originally intended/described here. I think it's okay to use 401 as a response to a XHR request, and then have the client redirect to a login page, which provides a way to authenticate again (reattempt the authentication challenge), analogous to how it works for HTTP authentication.

Revised opinion

https://stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses/14713094#14713094 has made me change my mind and convinced me that...

Authentication by schemes outside of (not defined by) RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication should not use HTTP status 401, because 401 Unauthorized is only defined (by current RFCs) by RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication, and has semantics and requirements (such as the requirement that "A server generating a 401 (Unauthorized) response MUST send a WWW-Authenticate header field containing at least one challenge.") that simply don't make sense or cannot be fulfilled if using a non-HTTP authentication scheme.

403 Forbidden, on the other hand, is defined by the broader HTTP standard, in RFC7231: Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content and RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication.

In conclusion, if you have your own roll-your-own login process and never use HTTP Authentication, 403 is always the proper response and 401 should never be used.

Couldn't a custom auth system use WWW-Authenticate header?

The question was asked:

Doesn't RFC7235 provide for "roll-your-own" or alternate auth challenges? Why can't my app's login flow present its challenge in the form of a WWW-Authenticate header? Even if a browser doesn't support it, my React app can...

And I would say sure, if you want (and if the browser doesn't automatically show a Basic auth modal in this case and thwart your plans).

They might be on to something here with that question!

But that should probably be the test of whether you can/should use 401: are you actually using WWW-Authenticate header?

Indeed I found an example where it is used for OAuth2.

What does "are not considered when deciding whether to use 401 or 403" mean exactly? What exactly should not be considered, and what exactly should be considered instead? In other words, how did someone arrive at the conclusion that "if you have your own roll-your-own login process and never use HTTP Authentication, 403 is always the proper response and 401 should never be used."? Why is 403 okay to use for non-HTTP authentication, but not 401?

Oh, I think I understand the difference now.

They should have said:

Authentication by schemes outside of (not defined by) RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication should not use HTTP status 401, because 401 Unauthorized is only defined (by current RFCs) by RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication, and has semantics and requirements (such as the requirement that "A server generating a 401 (Unauthorized) response MUST send a WWW-Authenticate header field containing at least one challenge.") that simply don't make sense or cannot be fulfilled if using a non-HTTP authentication scheme.

403 Forbidden, on the other hand, is defined by the broader HTTP standard, in RFC7231: Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content and RFC7235: Hypertext Transfer Protocol (HTTP/1.1): Authentication.

In conclusion, if you have your own roll-your-own login process and never use HTTP Authentication, 403 is always the proper response and 401 should never be used.

See also my comments in https://hyp.is/p1iCnnowEeyUPl9PxO8BuQ/www.rfc-editor.org/rfc/rfc7235

See my comments in https://hyp.is/p1iCnnowEeyUPl9PxO8BuQ/www.rfc-editor.org/rfc/rfc7235

Moved this comment to: https://hyp.is/p1iCnnowEeyUPl9PxO8BuQ/www.rfc-editor.org/rfc/rfc7235