Top Google results for [anonymous EA criticism]
Similar to what I was imagining, though I'd post them in a more linkable-to place so you could tell your friends to fix specific issues
arikr in 2019 https://forum.effectivealtruism.org/posts/k8d59pe4zryfaioM6/link-what-opinions-do-you-hold-that-you-would-be-reluctant
Also related, though the authors chose who to ask and who they were asking wasn't anonymous to them
Rob Bensinger in 2017 https://forum.effectivealtruism.org/posts/QYGNmzqjhHgxfbyhy/anonymous-ea-comments
80,000 hours in 2020 https://80000hours.org/author/anonymous/