PrecisionBiome.EU, who is using the Microbiome Prescription suggestion engine (with minor customization), ask me to update the engine’s Cross Validation statistics. They are doing Shotgun analysis in conformity to all EU laws and regulations. It has been almost 2 years since I posted Cross Validation of AI Suggestions for Nonalcoholic Fatty Liver Disease which had 92% agreement. As you can see below, we are now at 97% for NAFLD.
Cross validation means computing suggestions from the typical shifts reported on PubMed. For simple validation, we do a new count of the highs and lows. Some studies report a bacteria being high, a different study reported it being low for the same condition — both claiming statistical significant. I term this data to be fuzzy.
We then run the resulting shifts through the suggestion engine and get two numbers:
- Matches — suggestions says to take and studies report that it helps the condition (for some people in the study)
- Not Matches — suggestions says to avoid and studies report that it helps the condition (for some people in the study)
Again, we see different results reported in the same study, we have more fuzzy data. Last item is that we need a sufficient number of substances known to help the condition to reduce randomness.
The results are below. My criteria starting Microbiome Prescription was to do better than random for suggestions. Random would be 50%. We exceed that.
| ConditionName | Matches | Not Matches | Percentage Correct |
| Depression | 65 | 7 | 90% |
| Irritable Bowel Syndrome | 46 | 7 | 87% |
| Biofilm | 39 | 0 | 100% |
| Chronic Fatigue Syndrome | 37 | 18 | 67% |
| Crohn’s Disease | 35 | 7 | 83% |
| High Histamine/low DAO | 35 | 2 | 95% |
| hypertension (High Blood Pressure | 33 | 5 | 87% |
| Nonalcoholic Fatty Liver Disease (nafld) Nonalcoholic | 30 | 1 | 97% |
| Obesity | 30 | 3 | 91% |
| Autism | 28 | 7 | 80% |
| Inflammatory Bowel Disease | 24 | 1 | 96% |
| Type 2 Diabetes | 24 | 3 | 89% |
| obsessive-compulsive disorder | 24 | 3 | 89% |
| Allergic Rhinitis (Hay Fever) | 21 | 1 | 95% |
| Ulcerative colitis | 20 | 9 | 69% |
| Asthma | 18 | 8 | 69% |
| Constipation | 19 | 2 | 90% |
| Brain Trauma | 15 | 10 | 60% |
| Metabolic Syndrome | 14 | 0 | 100% |
| Stress / posttraumatic stress disorder | 14 | 1 | 93% |
| Mast Cell Issues / mastitis | 13 | 0 | 100% |
| Functional constipation / chronic idiopathic constipation | 13 | 3 | 81% |
| Cancer (General) | 12 | 5 | 71% |
| Overall | 609 | 103 | 86% |
Let us look at the poor performers above:
- Chronic Fatigue Syndrome has a complex set of causes: COVID, Lyme Disease, Food Poisoning etc.
- Brain Trauma can be from many causes also, having a wide variation
- Cancer – this is also a big collection of things
From the point of view of most Artificial Intelligence methodologies, 86% is very good.
Bottom Line
None of the above conditions are “clean”, in general there can be a multitude of subsets for each condition; each subset likely have different microbiome shifts. For every modifier, you have a percentage of responders, non-responders and adverse events. Furthermore, diet styles of the patients impacts bacteria shifts. In short, all of the data is fuzzy. This is not a concern for me because I have several decades dealing with fuzzy logic issues professionally.
P.S. IMHO, this is the best way of producing evidence that any suggestion engine are producing reasonable results. The usual clinical study methodology just does not work.
Recent Comments