It seems a fair AI would treat you the same if you had a different race/gender/disability/etc., but how can we ever test counterfactual fairness? In #NeurIPS2023 w Victor Veitch, we show you sometimes can with simple, observed metrics like group parity! https://arxiv.org/abs/2310.19691