If I am understanding this, I don't agree with your conclusion that "vaccinations alone do not explain this change in trends." The reason I say this is that your second graph show that excess infant mortality clearly increased after mother and child vaccinations. That is all you need to see to say they are related. The state data being scattered with no apparent trends is a more complicated issue.
When I say "explain", I mean explain through a mathematical model, like a linear regression. In a simple linear regression there is just one explanatory variable, in this case "vaccinations alone".
How well vaccinations alone can explain the trend change can be measured, e.g. with R-Squared. The explanatory power of the model is less than 1%.
So no, vaccinations alone do not explain this situation, mathematically speaking.
Nice work Fabian. The graphical evidence is compelling but it could be a causal vax signal gets lost when aggregating by state and year. I think it is worth staying at the county level by month and regressing on known large effects like median household income, population, and month of year and also include lags of vax update.
Don't you think infant mortality data are too sparse to get meaningful results from monthly county-level data?
What I am doing right now is trying to understand what diagnostic keys saw upticks in 2022 on national level (annual). So I've done the exact opposite to what you suggested and zoomed out for now. (Some absolutely shocking crystal-clear signals there, which I will present in Part 2, hopefully today).
Then maybe when I have a better understanding of the shape of mortality I'd like to understand what impact vaccinations had.
The problem with county-level data are the horrible population estimates. This is not an issue when we have older reference data to use as the denominator (e.g. for deaths), but when it comes to vaccination rates/coverage, I have to use population as the denominator.
Alas both accuracy of population estimates and variance in vaccination coverage are quite low. I've been wondering how I could take this into account.
As usual, it's good to try many different analyses to gain the best possible handle on what is happening in the data at various resolutions. Simpson's paradox cases become more of a danger at higher levels of aggregation.
There are techniques like zero-inflated Poisson (ZIP) regression that can handle data with a lot of zeros.
Although US citizens do like to move somewhat frequently, my sense is population estimates and other important measures like average socioeconomic status for each county should be somewhat stable and obtainable.
Accurate estimates of vax uptake is another matter and getting those at a county level might be quite difficult. Using state-level numbers for them could be the best we can do and at least a reasonable approximation.
These signals are absolutely shocking. I'll send you an email with a few examples, so you get an idea.
County-level population estimates are off by up to 100%/50%. Apache county is a good example for this, but there's an array of counties that are similarly off.
I've plotted county-level vaccination coverage and cumulative cases per capita as a sorted bar chart which makes this very clear.
Using these estimates introduces major bias into all correlations, unless there's large variance in the numerator (which there isn't in vaccination data).
I found the best way to deal with this was just to remove all counties that have exceptionally high or low cumulative cases/doses per capita, but maybe there's a better way.
CDC Wonder. I just combine requests. For infant mortality I just picked age groups <1, <1 - 100, 1 - 100, and combined the results to get as much monthly data stratified by ICD-10 subchapter (MCOD) as I can.
Getting weekly U07.1 time series stratified by vaccination age required making around 300 requests.
CDC Wonder is just an amazing resource if you are willing to go through this trouble.
Zowe substack. She writes about how codes were changed or bunched under the new Covid one. Also mentioned was how new codes relating to the covid trials had no codes already set up as was norm with trials. Then Zowe writes WHO did set up code forAE but this was not communicated to coders. Only 18 records had this code. All to make it very difficult for guys like you to pull data out to easily analyse.
If I am understanding this, I don't agree with your conclusion that "vaccinations alone do not explain this change in trends." The reason I say this is that your second graph show that excess infant mortality clearly increased after mother and child vaccinations. That is all you need to see to say they are related. The state data being scattered with no apparent trends is a more complicated issue.
No, that is just a coincidence by definition.
When I say "explain", I mean explain through a mathematical model, like a linear regression. In a simple linear regression there is just one explanatory variable, in this case "vaccinations alone".
How well vaccinations alone can explain the trend change can be measured, e.g. with R-Squared. The explanatory power of the model is less than 1%.
So no, vaccinations alone do not explain this situation, mathematically speaking.
An article a day keeps the doctor away.
You got it! :D
Nice work Fabian. The graphical evidence is compelling but it could be a causal vax signal gets lost when aggregating by state and year. I think it is worth staying at the county level by month and regressing on known large effects like median household income, population, and month of year and also include lags of vax update.
Thank you, Russ.
Don't you think infant mortality data are too sparse to get meaningful results from monthly county-level data?
What I am doing right now is trying to understand what diagnostic keys saw upticks in 2022 on national level (annual). So I've done the exact opposite to what you suggested and zoomed out for now. (Some absolutely shocking crystal-clear signals there, which I will present in Part 2, hopefully today).
Then maybe when I have a better understanding of the shape of mortality I'd like to understand what impact vaccinations had.
The problem with county-level data are the horrible population estimates. This is not an issue when we have older reference data to use as the denominator (e.g. for deaths), but when it comes to vaccination rates/coverage, I have to use population as the denominator.
Alas both accuracy of population estimates and variance in vaccination coverage are quite low. I've been wondering how I could take this into account.
As usual, it's good to try many different analyses to gain the best possible handle on what is happening in the data at various resolutions. Simpson's paradox cases become more of a danger at higher levels of aggregation.
There are techniques like zero-inflated Poisson (ZIP) regression that can handle data with a lot of zeros.
Although US citizens do like to move somewhat frequently, my sense is population estimates and other important measures like average socioeconomic status for each county should be somewhat stable and obtainable.
Accurate estimates of vax uptake is another matter and getting those at a county level might be quite difficult. Using state-level numbers for them could be the best we can do and at least a reasonable approximation.
Thank you, I will look into ZIP!
These signals are absolutely shocking. I'll send you an email with a few examples, so you get an idea.
County-level population estimates are off by up to 100%/50%. Apache county is a good example for this, but there's an array of counties that are similarly off.
I've plotted county-level vaccination coverage and cumulative cases per capita as a sorted bar chart which makes this very clear.
Using these estimates introduces major bias into all correlations, unless there's large variance in the numerator (which there isn't in vaccination data).
I found the best way to deal with this was just to remove all counties that have exceptionally high or low cumulative cases/doses per capita, but maybe there's a better way.
> downloaded a bunch of data
Source link?
CDC Wonder. I just combine requests. For infant mortality I just picked age groups <1, <1 - 100, 1 - 100, and combined the results to get as much monthly data stratified by ICD-10 subchapter (MCOD) as I can.
Getting weekly U07.1 time series stratified by vaccination age required making around 300 requests.
CDC Wonder is just an amazing resource if you are willing to go through this trouble.
Have you seen the articles from the now ex medical coder? Will need to find you link if you are not aware.
Which one?
Zowe substack. She writes about how codes were changed or bunched under the new Covid one. Also mentioned was how new codes relating to the covid trials had no codes already set up as was norm with trials. Then Zowe writes WHO did set up code forAE but this was not communicated to coders. Only 18 records had this code. All to make it very difficult for guys like you to pull data out to easily analyse.
Yes will do. It's possible I overlooked it. Will check when I'm home.