This article explores different approaches to assessing the effectiveness of non-state-based non-judicial grievance mechanisms (NSBGMs) in achieving access to remedy for rightsholders. It queries the approach that has been widely adopted as a result of the United Nations Guiding Principles on Business and Human Rights (UNGPs), which focuses on the procedural aspects of grievance mechanisms. Rather, it stresses the importance of analysing the outcomes of cases for rightsholders. This article tests this hypothesis by undertaking comprehensive empirical research into the complaint mechanism of the Roundtable on Sustainable Palm Oil (RSPO). RSPO is found to perform well when judged according to the UNGPs’ effectiveness criteria. However, it performs poorly when individual cases are assessed to ascertain the outcomes that are achieved for rightsholders. The article therefore argues for the importance of equivalent scrutiny of outcomes in relation to other NSBGMs and provides an approach and accompanying methodology that can be utilized for that purpose.