Related talks: 2012.03.20 Bernstein. 2012.04.17 Bernstein. 2012.10.30 Bernstein. 2013.01.15 Bernstein--Lange. 2013.02.10 Bernstein. 2013.02.10 Lange.
Publication of ideas is a fundamental part of the academic world. Publication of ideas is emphasized in academia because it is vital to the scientific method. This, on a basic level, means that if you have an idea, you toss it out into the "marketplace of ideas" and your peers and others evaluate it, test it and discuss it. This is how we determine which ideas are good and which are faulty. ---Andrew W. Appel, 1996Sounds great, but the current reality is that many academics pay very little attention to publications that haven't been "accepted" by small panels of reviewers. Typically (at least in mathematics and computer science) a panel consists of only three reviewers for a conference, and only two reviewers for a journal; often one or more reviewers have only limited expertise in the area. The reviewers don't carry out their discussions in public. What happens when the reviewers' ideas are faulty? Where is the public evaluation and testing of those ideas?
We've received an amazing collection of reviews for "Non-uniform cracks in the concrete: the power of free precomputation". Some reviews are positive; some reviews are negative; but what's incredible is the overall level of sloppiness of the reviewing process, with many reviews full of faulty ideas that would never survive broader examination. Here are the reviews: