More than 100 leading experts and independent AI evaluators have gathered as part of the AI Evaluators Forum consortium to highlight the lack of resources and safeguards to adequately test advanced models. In an open letter signed by Geoffrey Hinton, experts asked the creators of the underlying model to ensure scientific objectivity, transparency and protection against possible retaliation or litigation.
The reason for the discussion was a recent proposal to provide independent researchers with privileged access to proprietary developments and internal company systems. However, experts say true audit independence requires clear standards that eliminate conflicts of interest, guarantee free release of results and protect auditors from pressure from tech giants.