New Feature: Over and Under Representation

No, I am not talking about voting politics in the US!

While doing an analysis, I went to the raw data to try to understand the sample. The result is the addition of a new section on the [Research Features] tab. Unlike most items, this is not directly actionable. An analogy:

You have gotten 100 used coins from the bank and proceeded to toss each one once. You would expect to get 50 heads and 50 tails. You got 20 heads and 80 tails. This means that these 100 coins have bias that is statistically significant. You do not know which are the problem (unfair) coins.

The same issue applied to vectors of the microbiome.

A reader had just emailed me that they have done another sample and it occur to me to view a time series of this person over time to see what this new report offers. The person reports some improvements following Dr. Artificial Intelligence suggestions. I included Dr. Jason Hawrelak rating on each for reference

Nov 21, 2021, Jason: 56%ile
March 15,2022, Jason: 95.6%
May 16, 2022, Jason: 89%ile
June 15, 2022, Jason: 89%ile

The biggest improvement with Dr. Jason Hawrelak was between the first two. KEGG Compounds went from being under produced for both high and low, to over on all subsequent ones. The pattern of over and under kept consistent until the very last one where bacteria edged into significance. I do have concerns with single digit Z-Scores, because of the false discovery rate.

What does Over Representation of Low Bacteria mean exactly? It means that the number of different bacteria types sitting below 10% was much higher than expected. It may imply a more diverse population with a lot of token representation.

What does Under Representation of High Bacteria mean exactly? It’s the flip side of above. The number of different bacteria types sitting above 90% was much lower than expected. It may imply a population without full representation.

WARNING: Do not assign undue significance to a change of z-score with the same sign.

On a personal note, seeing bacteria shift into significance from insignificance, looks like a good thing. It means that the prior microbiome has become disrupted. Our goal is to disrupt the stable dysfunctional microbiome causing symptoms.

Again, this is both an experimental feature AND it’s interpretation is not easy.