| 1. | | I'm on a deserted island. How can I tell which plants are poisonous? (straightdope.com) |
| 104 points by yanowitz on Jan 22, 2010 | 62 comments |
|
| 2. | | Keeping computers from ending science's reproducibility (arstechnica.com) |
| 101 points by yummyfajitas on Jan 22, 2010 | 62 comments |
|
| 3. | | Stop restricting my password - Help these sites get better security. (weakpasswords.org) |
| 84 points by dinkumator on Jan 22, 2010 | 60 comments |
|
| 4. | | Rails and Merb Merge Update: Rails Core (engineyard.com) |
| 84 points by wifelette on Jan 22, 2010 | 3 comments |
|
| 5. | | WSJ Jumps the Shark (ritholtz.com) |
| 81 points by tortilla on Jan 22, 2010 | 67 comments |
|
| 6. | | API Status (api-status.com) |
| 76 points by jmonegro on Jan 22, 2010 | 8 comments |
|
| 7. | | PDFs in Pure Ruby (majesticseacreature.com) |
| 67 points by jmonegro on Jan 22, 2010 | 46 comments |
|
| 8. | | The jQuery Project launched (jquery.org) |
| 60 points by ronnier on Jan 22, 2010 | 14 comments |
|
| 9. | | Hustle (how to learn new stuff) (ihumanable.com) |
| 60 points by ihumanable on Jan 22, 2010 | 10 comments |
|
| |
|
|
| 11. | | How to sue your employer and win (unposto.wordpress.com) |
| 58 points by tallyh00 on Jan 22, 2010 | 22 comments |
|
| 12. | | SF Mayor on why open source is the new software policy in San Francisco (mashable.com) |
| 55 points by anigbrowl on Jan 22, 2010 | 27 comments |
|
| |
|
|
| 14. | | Google Chrome's H.264 support not true "free" software (ianweller.org) |
| 54 points by cpearce on Jan 22, 2010 | 82 comments |
|
| |
|
|
| 16. | | OfficePod - Tiny, Minimalist Office Space (officepod.co.uk) |
| 48 points by rgrieselhuber on Jan 22, 2010 | 53 comments |
|
| 17. | | Lisp Quotes (paulgraham.com) |
| 47 points by alrex021 on Jan 22, 2010 | 50 comments |
|
| 18. | | Vijual Graph Layout Library For Clojure (lisperati.com) |
| 46 points by swannodette on Jan 22, 2010 | 6 comments |
|
| 19. | | Steve Jobs Is Building AppleWorld - And Google's Running Scared (seekingalpha.com) |
| 46 points by profquail on Jan 22, 2010 | 76 comments |
|
| 20. | | The Chess Master and the Computer (by Garry Kasparov) (nybooks.com) |
| 44 points by michael_nielsen on Jan 22, 2010 | 5 comments |
|
| 21. | | Ban on unscientific "bomb detector" after $85M sales (bbc.co.uk) |
| 44 points by jodrellblank on Jan 22, 2010 | 28 comments |
|
| 22. | | Committing Location Based Service Suicide (andrewhy.de) |
| 44 points by davidhoffman on Jan 22, 2010 | 28 comments |
|
| 23. | | JavaScript grid editor: I want to be Excel. Updated (open-lab.com) |
| 42 points by alake on Jan 22, 2010 | 7 comments |
|
| |
|
|
| 25. | | Data Mining competition for predicting drug reactions (orwik.com) |
| 40 points by bucanrabi on Jan 22, 2010 | 4 comments |
|
| 26. | | Explore GitHub (trending/featured repos and podcasts) (github.com/explore) |
| 39 points by pjhyett on Jan 22, 2010 | 10 comments |
|
| |
|
|
| 28. | | Tell HN: I released my open source iPhone AppStore Sales Graphing Tool (maxklein.github.com) |
| 38 points by maxklein on Jan 22, 2010 | 14 comments |
|
| 29. | | Microsoft Reveals the Science Behind Project Natal for Xbox 360 (scientificamerican.com) |
| 38 points by stakent on Jan 22, 2010 | 23 comments |
|
| 30. | | Passive solar glass home: watching the sun move (faircompanies.com) |
| 37 points by kirstendirksen on Jan 22, 2010 | 5 comments |
|
|
| More |
It seems like the vast bulk of these simulations are iterative, and therefore subject to mathematical chaos. How many of these researchers have any clue what door they are walking through? How many of them know what a strange attractor is? I'm sure the answer is non-zero; I'm equally sure the answer is nowhere near 100%.
Small errors cascade even if you consider a non-chaotic classical model. (That is, not that there is such a thing as an iterative model that is not potentially subject to chaos, but rather than even if you don't understand chaos you can see that small errors can cascade. Chaos just makes it worse, and weirder.) A simulation will have bugs like any other large problem. A non-programmer approaches bugs by banging on the program until it seems to generate expected results. (About 50-80% of programmers do that too.) Therefore, many of these simulations are simply reflections of the simulator's expected result, due to the effect of the researcher's selection mechanism running on the results of the simulations they run. How do we verify that this is not the primary factor in the result of the simulation? This need not be conscious. It need not be ideological, either; I can easily envision a simulation that "should" return a boring or trivial result being monkeyed with until it produces something "interesting", because the simulators think the boring result should not obtain.
A lot of algorithms you can use in these simulations are fundamentally unstable when used iteratively; some exacerbated by floating point errors, some mathematically unstable even with perfect real numbers. How many of these simulations use something unstable without even realizing it, given that it could take a professional mathematician to work out whether that's the case? Even algorithms thought to be stable and reliable can fall apart under pathological situations, and one of the odd things about mathematics is just how often you end up hitting those pathological situations when programming; far more often than it seems like should be the case.
In information theory terms, a simulation can not contain more information that the sum total of the input data and the content of the simulation algorithm. How many simulators understand the full implications of that statement? I sure don't understand the full implications of that, but what I do understand makes me pause a bit. Very simple simulations with rules that can be verified and initial data that is very solid I can deal with; for instance, I like the cell-automata based social theories that show the spread of information or political views or something, especially when it is clear the researchers understand that it's only an approximation. But as the initial data starts getting sketchy or the simulation grows enormous, I start getting nervous about the actual information content of the output. Just because the output appears to be information doesn't prove that it is. It is vitally necessary to be able to check the simulation against real data. For instance, physical simulations of, say, cars crashing can be verified. How many simulations can actually be verified, though? Frequently the reason computers were reached for in the first place is the inability to do the real experiment. Any simulation that can't be verified should be presumed worthless by default. How often does that happen? (It's 20-f'ing-10 and "the computer said it, it must be right" still runs rampant through our culture....)
And of course there's the whole reproducibility issue, where the absolute bare minimum for science would be to publish the full simulation program, all data, the necessary invocation and compile instructions to bring the two together, and all necessary information to understand the input and the output. Clearly, this is not something that fits in a journal paper, but how often does this happen at all?
No, I am not referring to any specific discipline here and in particular I'm not actually referring to climate science. I'm nervous about the whole movement towards simulations in general.
Note that I'm not reflexively against the idea. Meet these bars and I'm happy; give me enough data for reproducibility and verify that your simulation is in fact simulating something real and corresponds to reality and I am happy. (Many physical simulations fit in here.) But as more disciplines jump in I am concerned that these bars are not well understood, and I'm seeing ever more press releases about simulations that can't possibly meet these bars.