Covid-sim: Remove 23 people from Alaska (2020)
1 points · Open discussionmrc-ide
covid-sim
C++2020–2022
mrc-ide / covid-sim
covid-sim
This is the COVID-19 CovidSim microsimulation model developed by the MRC Centre for Global Infectious Disease Analysis hosted at Imperial College, London.
27Distinct people
34Mentions
0Explicit recommenders
1.2KGitHub stars
What to know before exploring
What it is
This is the COVID-19 CovidSim microsimulation model developed by the MRC Centre for Global Infectious Disease Analysis hosted at Imperial College, London.
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month34 dated mentions
Observed mentions: 2020-04-27 20:42:08 UTC to 2022-08-03 02:26:38 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Apr 2020 | 2020-04-01 00:00:00 UTC | 2020-05-01 00:00:00 UTC | 5 | 3 | 2 |
| May 2020 | 2020-05-01 00:00:00 UTC | 2020-06-01 00:00:00 UTC | 22 | 5 | 17 |
| Jun 2020 | 2020-06-01 00:00:00 UTC | 2020-07-01 00:00:00 UTC | 2 | 0 | 2 |
| Jul 2020 | 2020-07-01 00:00:00 UTC | 2020-08-01 00:00:00 UTC | 0 | 0 | 0 |
| Aug 2020 | 2020-08-01 00:00:00 UTC | 2020-09-01 00:00:00 UTC | 1 | 0 | 1 |
| Sep 2020 | 2020-09-01 00:00:00 UTC | 2020-10-01 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2020 | 2020-10-01 00:00:00 UTC | 2020-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2020 | 2020-11-01 00:00:00 UTC | 2020-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2020 | 2020-12-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2021 | 2021-01-01 00:00:00 UTC | 2021-02-01 00:00:00 UTC | 1 | 0 | 1 |
| Feb 2021 | 2021-02-01 00:00:00 UTC | 2021-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2021 | 2021-03-01 00:00:00 UTC | 2021-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2021 | 2021-04-01 00:00:00 UTC | 2021-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2021 | 2021-05-01 00:00:00 UTC | 2021-06-01 00:00:00 UTC | 2 | 0 | 2 |
| Jun 2021 | 2021-06-01 00:00:00 UTC | 2021-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2021 | 2021-07-01 00:00:00 UTC | 2021-08-01 00:00:00 UTC | 0 | 0 | 0 |
| Aug 2021 | 2021-08-01 00:00:00 UTC | 2021-09-01 00:00:00 UTC | 0 | 0 | 0 |
| Sep 2021 | 2021-09-01 00:00:00 UTC | 2021-10-01 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2021 | 2021-10-01 00:00:00 UTC | 2021-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2021 | 2021-11-01 00:00:00 UTC | 2021-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2021 | 2021-12-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2022 | 2022-01-01 00:00:00 UTC | 2022-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2022 | 2022-02-01 00:00:00 UTC | 2022-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2022 | 2022-03-01 00:00:00 UTC | 2022-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2022 | 2022-04-01 00:00:00 UTC | 2022-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2022 | 2022-05-01 00:00:00 UTC | 2022-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2022 | 2022-06-01 00:00:00 UTC | 2022-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2022 | 2022-07-01 00:00:00 UTC | 2022-08-01 00:00:00 UTC | 0 | 0 | 0 |
| Aug 2022 | 2022-08-01 00:00:00 UTC | 2022-09-01 00:00:00 UTC | 1 | 1 | 0 |
From the project README
An excerpt from the repository, fetched Nov 13, 2026.
Read the project’s introduction
COVID-19 CovidSim Model
This is the COVID-19 CovidSim microsimulation model developed by the MRC Centre
for Global Infectious Disease Analysis hosted at Imperial College, London.
CovidSim models the transmission dynamics and severity of COVID-19 infections
throughout a spatially and socially structured population over time. It enables
modelling of how intervention policies and healthcare provision affect the
spread of COVID-19. It is used to inform health policy by making quantitative
forecasts of (for example) cases, deaths and hospitalisations, and how these will
vary depending on which specific interventions, such as social distancing, are enacted.
With parameter changes, it can be used to model other respiratory viruses,
such as influenza.
IMPORTANT NOTES
⚠️ This code is released with no support.
⚠️ This model is in active development and so parameter name and
behaviours, and output file formats will change without notice.
⚠️ The model is stochastic. Multiple runs with different seeds should be
undertaken to see average behaviour. This can now be done easily with the /NR
command line parameter. The model code behaves deterministically if run with the
same number of threads enabled and run with the same random number seeds.
⚠️ As with any mathematical model, it is easy to misconfigure inputs and
therefore get meaningless outputs. The Imperial College COVID-19 team only
endorses outputs it has itself generated.
Status
This model is in active development and subject to significant code changes
to:
Enable modelling of more geographies
Enable modelling of different intervention scenarios
Improve performance
Building
The model is written in C++.
The primary platforms it has been developed and tested on are Windows and Ubuntu Linux.
It should build and run on any other POSIX compliant Unix-like platform (for example macOS, other Linux distributions). However, no active development occurs on them.
Running the model for the whole of the UK requires approximately 20GB of RAM.
Other regions will require different amounts of memory (some up to 256GB).
It is strongly recommended to build the model with OpenMP support enabled to
improve performance on multi-core processors. 24 to 32 core Xeon systems give
optimal performance for large (e.g. UK, US) populations.
See build.md for detailed build instructions.
Testing
From within your build directory do:
make test
# If you want more progress indication
make test ARGS="-V"
# To parallelise tests add a -jN option for instance:
make test ARGS="-j6"
make test ARGS="-j6 -V"
# or
ctest -V
ctest -V -j6
# etc...
IMPORTANT: The test scripts use test data only are not runs reflective of
real-world situations.
Sample Data
The directory data contains sample data.
The Python script run_sample.py demonstrates how to
invoke CovidSim to use this data. See the sample README for
details on how to run the samples.
The directory report9 contains files to allow the results tables in the Imperial College
Report 9 - Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand
to be reproduced.
Documentation
Model documentation can be found in the docs directory. Of
particular interest are:
Model Overview
Model Glossary
Model Inputs and Outputs
Interventions description
R Scripts for Output Visualisation
Given the entire Imperial College team is working full-time on the COVID-19
response, documentation is currently sparse. More documentation and sample files
will be added as time permits. In the coming few weeks this will include a much
more extensive set of input files to model strategies for exiting lockdown.
Relevant papers
The following papers are relevant to the model. Please note that some of them
may require a subscription.
https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf
https://www.nature.com/articles/nature04795
https://www.nature.com/articles/nature04017
https://www.pnas.org/content/105/12/4639.short
Copyright and Licensing
The source code for CovidSim is licensed under the GPLv3, see
LICENSE.md.
It is Copyright Imperial College of Science, Technology and Medicine. The
lead developers are Neil Ferguson, Gemma Nedjati-Gilani and Daniel Laydon.
Additional contributions for open-sourcing made by Imperial College of
Science, Technology and Medicine, GitHub Inc, and John Carmack are copyright
the authors.
Licensing details for material from other projects may be found in
NOTICE.md. In summary:
CovidSim includes code modified from
RANLIB which
is licensed under the LGPLv3.
Sample data in the repository has been derived from the following sources:
WorldPop (www.worldpop.org - School of Geography and Environmental Science,
University of Southampton; Department of Geography and Geosciences, University
of Louisville; Departement de Geographie, Universite de Namur) and Center for
International Earth Science Information Network (CIESIN), Columbia University
(2018). Global High Resolution Population Denominators Project - Funded by The
Bill and Melinda Gates Foundation (OPP1134076).
https://dx.doi.org/10.5258/SOTON/WP00647
WorldPop is licensed under the Creative Commons Attribution 4.0 International
License (CC BY 4.0). The text of the license can be found at:
https://creativecommons.org/licenses/by/4.0/legalcode
Contributing
Due to time pressure on the development team, we are unable to provide user
support at this time.
If you find issues with the code please raise them in our
Issue Tracker.
This repository has a code of conduct which is detailed in
the code of conduct. When raising an issue in this
repository you agree to abide by the code of conduct.
Read the full README on GitHubWhat Hacker News says
34 sourcesError bars would be nice. They're MIA in large swathes of COVID related research. I've read a lot of COVID papers in the past year and this paper is typical of the field. Things you should expect to see when reading epidemiology literature:
1. Statistical uncertainty is normally ignored. They can and will tell politicians to adopt major policy changes on the back of a single dataset with 20 people in it. In the rare cases when they bother to include error bars at all they are usually so wide as to be useless. In many other fields researchers debate P-hacking and what threshold of certainty should count as a significant finding. Many people observe that the standard of P=0.05 in e.g. psychology is too high because it means 1 in 20 studies will result significant-but-untrue findings by chance alone. Compared to those debates epidemiology is
Read the rest
in the stone age: any claim that can be read into any data is considered significant.
2. Rampant confusion between models and reality. The top rated comment on this thread observes that the paper doesn't seem to test its model predictions against reality yet makes factual claims about the world. No surprises there; public health papers do that all the time. No-one except out-of-field skeptics actually judge epidemiological models by their predictive power. Epidemiologists admit this problem exists, but public health has become so corrupt that they argue being able to correctly predict things is not a fair way to judge a public health model[1]. Obviously they insist governments should still implement whatever policies the models say are required. It's hard to get more unscientific than culturally rejecting the idea that science is about predicting the natural world, but multiple published papers in this field have argued exactly that. A common trick is "validating" a model against other models [2].
3. Inability to do maths. Setting up a model with reasonable assumptions is one thing but do they actually solve the equations correctly? The Ferguson model from Imperial College, which we're widely assured is one of the world's top teams of epidemiologists, was written in C and filled with race conditions/out of bounds reads that caused their model to totally change its predictions due to timing differences in thread scheduling, different CPUs/compilers etc. These differences were large, e.g. a difference of 80,000 deaths predicted by May for the UK [3]. Nobody in the academic hierarchy saw any problem with this and worse, some researchers argued that such errors didn't matter because they just ran it a bunch of times and averaged the results. This is confusing the act of predicting the behaviour of the world with the act of measuring it, see point (2).
4. Major logic errors. Assuming correlation implies causation is totally normal. Other fields use sophisticated approaches to try and control for confounding variables, epidemiology doesn't. Circular logic is a lot more common than normal, for some reason.
None of these problems stop papers being published by supposedly reputable institutions in supposedly reputable journals. After reading or scan-reading about 50 epidemiology papers, including some older papers from 10 years ago, I concluded that not a single thing from this field can be trusted. The problems aren't specific to COVID, they're cultural and have been around a long time. Life is too short to examine literally every paper making every claim but if you take a sample and nearly all of them contain basic errors or what is clearly actual fraud, then it seems fair to conclude the field has no real standards.
[1] "few models in healthcare could ever be validated for predictive use. This, however, does not disqualify such models from being used as aids to decision making ... Philips et al state that since a decision-analytic model is an aid to decision making at a particular point in time, there is no empirical test of predictive validity. From a similar premise, Sculpher et al argue that prediction is not an appropriate test of validity for such model" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3001435/
[2] https://github.com/ptti/ptti/blob/master/README.md
[3] https://github.com/mrc-ide/covid-sim/issues/116 https://github.com/mrc-ide/covid-sim/issues/30 https://github.com/mrc-ide/covid-sim/commit/581ca0d8a12cddbd... https://github.com/mrc-ide/covid-sim/commit/3d4e9a4ee633764c...
I'd suggest reading some scientific papers before going too far in that direction. I've read a lot of them. These scientists routinely do things like cite papers that contradict the claim the citation is meant to support. They write papers based on circular logic. They conflate simulations with reality, they make claims that have no basis in biological science. They do not know what the false positive rate on COVID testing is, but are happy to draw conclusions that flatly contradict decades of biological understanding on the basis of data analysis that critically depends on knowing the correct value. They write models in C that are filled with memory corruption bugs and race conditions. When the results randomly change from run to run they claim that it's an experiment and so averaging the results makes it all perfectly fine. And when pr
Read the rest
ofessional programmers tell them that incorporating memory corruption into their calculations is not fine, they assert those programmers don't understand science (see: https://github.com/mrc-ide/covid-sim/issues/165)
And that's just the scientists. Public health officials like Fauci have lied to the public and then admitted doing so, more than once! That isn't the data evolving, that's them deciding to change their story and explain that the previous one was a lie. What are we to make of such people?
You believe "experts know nothing" is insanity. I've read what those people are writing and am absolutely sure they know nothing, because if they did, they wouldn't keep engaging in these illegitimate, unscientific and desperately concerning practices.
"We in the World Health Organization do not advocate lockdowns as the primary means of control of this virus," Dr. David Nabarro (World Health Orginization).
You can find multiple articles that quote this from the WHO and link to the full 1 hour interview with the Spectator. The full interview goes on further to talk about how continued lockdowns will lead to move poverty and devastation.
There's also this study that tears apart the models used by many:
https://www.medrxiv.org/content/10.1101/2020.07.22.20160341v...
Also you know that 2-million Neil Ferguson model? Here's the source code for it. It's disastrous: non-reproducible results and 3 different random seeds. They literally ran it multiple times and took averages (with ranges of fatalities varying by up to 60k). Also it has race conditions and can't be run in parallel. Fucking mess
Read the rest
:
https://github.com/mrc-ide/covid-sim/blob/e8f7864ad150f40022...
As far as independent journalists that give you a different point of view:
James Corbett (who covers the origin of the lockdown policy): https://www.youtube.com/watch?v=nuDQ_3g53qc
Ben Shapiro makes an excellent case as well: https://www.youtube.com/watch?v=ymD9ipSm-eg
This article brings up scientific code from 10 years ago, but how about code from .. right now? Scientists really need to publish their code artifacts, and we can no longer just say "Well they're scientists or mathematicians" and allow that as an excuse for terrible code with no testing specs. Take this for example:
https://github.com/mrc-ide/covid-sim/blob/e8f7864ad150f40022...
This was used by the Imperial College for COVID-19 predictions. It has race conditions, seeds the model multiple times, and therefore has totally non-deterministic results[0]. Also, this is the cleaned up repo. The original is not available[1].
A lot of my homework from over 10 years ago still runs (Some require the right Docker container: https://github.com/sumdog/assignments/). If journals really care about the reproducibility crisis, artifact reviews need to be
Read the rest
part of the editorial process. Scientific code needs to have tests, a minimal amount of test coverage, and code/data used really need to be published and run by volunteers/editors in the same way papers are reviewed, even for non-computer science journals.
[0] https://lockdownsceptics.org/code-review-of-fergusons-model/
[1] https://github.com/mrc-ide/covid-sim/issues/179
At this point every scientist in the world is throwing any random theory out there in the hope one sticks and they will be showered in new funding. The track record of Neil Ferguson‘s Group at Imperial is representative of this industry, however.
https://github.com/mrc-ide/covid-sim/issues/176
In Initial Covid-19 infection rate may be 80 times greater than originally reported> why would the authors generally not include their code?
Decades of "printing out the code listings would cost too much paper" made many researchers get comfortable with the idea that nobody would ever see the code behind their work. That means:
- your code can be shit and potentially buggy and nobody will point and laugh at you
- replication is harder, so you can bluff some of the details
- if key findings are actually bugs, nobody will find out
These are all terrible reasons, but they're comfortable reasons as well. Open access was the first barrier, code included is simply the next.
If code included would've been commonplace, fiascos such as Imperial College of London's possibly-totally-bogus covid simulation [0][1] would've never made it to the cabinet.
[0] https://www.telegraph.co.uk/technology/2020/05/16/coding-led...
[1] https://g
Read the rest
We've perhaps not spent our time efficiently in this thread; it looks like they fixed the non-determinism a month ago,
https://github.com/mrc-ide/covid-sim/pull/121
Linked directly from the issue, too. It would indeed seem to have all been random numbers in the wrong order.
In Neil Ferguson's Imperial model could be the most devastating software mistakeI poked at the github repo for a bit. The ugliness of the code doesn't bother me, but the quantity of parameters does.
Here's one params file that specifies some of the inputs to a run of the model:
https://github.com/mrc-ide/covid-sim/blob/master/data/param_...
Here's another one:
https://github.com/mrc-ide/covid-sim/blob/master/data/admin_...
There are hundreds of constants in there. A lot of them appear to be wild-ass guesses. Presumably, all of them affect the output of the model in some way.
When a model has enough parameters for which you can make unsubstantiated guesses, you have a ton of wiggle room to generate whatever particular output you want. I'd like to see policy and public discussion focus more on the key parameters (R-naught, hospitalization rate, fatality rate) and less on overly-sophisticated models.
In To software engineers criticizing Neil Ferguson’s epidemics simulation codeHere's the GitHub issue:
https://github.com/mrc-ide/covid-sim/issues/116
The discussion isn't terribly clear to me. Is this non-determinism in which random number gets used where, like if you seed an RNG once and then use it from multiple threads that get scheduled in varying order? That wouldn't be good, but the model would still be performing as intended, just with no way to make it deterministic for testing. Or is it some other kind of non-determinism?
I'm not sure anyone knows, which indeed isn't totally comforting. I'm not sure the "bug uncertainty" is any bigger than all the existing known uncertainty about the assumptions that go in to his (or any such) model, though.
In Neil Ferguson's Imperial model could be the most devastating software mistakeIf anyone is in any doubt about this, I would like to point them towards this github issue: https://github.com/mrc-ide/covid-sim/issues/165
In It’s OK for your open source library to be a bit shitty (2015)The problem is gaining traction because it was used in the Covid 19 sim that was publicly released recently
https://github.com/mrc-ide/covid-sim/blob/master/README.md
There was even a thread about it that made its way to the front page of HN yesterday but was flagged without an explanation
https://news.ycombinator.com/item?id=23109211
In Monte Carlo Geometry Processing> The non-determinism that page talks about is coming from bugs like memory corruptions, ..., initialisation order bugs.
Source? I haven't seen these specifically cited anywhere.
> floating point inaccuracies,
Combined with (safe) race conditions, this will cause non-determinism that would probably be considered OK.
In general: John Carmack looked at the code and thought it was OK for what it is (decade old simulation transpiled from Fortran at some point). Some ex-Google guy thinks its horrible.
I looked at the code myself briefly. I haven't formed a strong opinion about the code myself beyond "it's ugly and I don't want to work with it, glad it's not my problem." I am however objecting to some of the comments here that make it sound like it is obviously broken for reasons that they just don't understand.
I think a comment on the GitHub
Read the rest
is relevant: https://github.com/mrc-ide/covid-sim/issues/175#issuecomment...
> To add to this, please read report 9 properly. The 500k UK prediction was if governments did nothing whatsoever - we never believed governments would do nothing but we modelled it as a base case, because that's part of what you do when you model.
> With the full social distancing the report suggestd it might be possible to reduce deaths perhaps to 20k - a death count we have already exceeded. Nobody here is laughing about that. The report was also very frank about the uncertainty involved in trying to predict what might happen at that stage
Yes, first find the serious issues and then take it down with vengeance. So far only serious issue is this:
https://github.com/mrc-ide/covid-sim/issues/168
Looks linked to the awful style, though. When you have 700 LOC functions starting with
int i, j, k, l, m, i1, i2, j2, l2, m2, tn;
it does tend to happen that you reuse a variable that you shouldn't have reused
In GitHub issue calling for retraction of Imperial College study for codebase flawsThere's some other criticism (which I won't link because the text is partisan, and it's elsewhere on HN already) with some links that point to (potential or perceived) issues:
- https://github.com/mrc-ide/covid-sim/issues/116
- https://github.com/mrc-ide/covid-sim/issues/30
In GitHub issue calling for retraction of Imperial College study for codebase flawsShowing the 15 most recent of 34 sources. Enable JavaScript to search and browse every indexed mention.