the review process is completely pointless with regards to reproducibility. the reviewers basically have to go off of what somebody wrote in the paper. other than maybe finding some systematic error from the writeup there is not really much a reviewer can actually detect and criticize (if the model works what else is there to say?). most published papers would be better off as just github projects with a proper descriptive readme that also shows benchmarks anyway. it’s not like papers are written very well to begin with. but that doesn’t get you a phd.
in physics there is basically no (or minimal) review process and publications are judged by how much your paper got cited. also, there is a full secondary track of researchers who just take other papers and recreate the experiments to actually confirm reproducibility. in ML right now there is no incentive for anyone to just run a published model on different/their own data and confirm that it works correctly. in fact you’d probably be crucified for doing that
Start training models
Go watch model trains
Come back to error