By Ken Lassesen, M.Sc.
Several years ago, I evaluated a broad range of statistical approaches for microbiome analysis, including machine-learning(ML) methods. I used publicly contributed samples available through the Microbiome Prescription citizen-science platform; the underlying dataset is available for download from its associated citizen-science repository.
At that time, I found that alternative statistical models generally outperformed the machine-learning approaches I tested. More recently, however, several direct-to-consumer microbiome testing companies have begun citing machine learning in their marketing materials. Time to revisit ML.
Perplexity summaries these as:
| Company | What it claims about ML/AI |
| Viome | Says its tests use metatranscriptomics and ML models in its scoring engine; it states those models turn microbial and biochemical patterns into Viome Scores and dietary/supplement recommendations. |
| ZOE | Says it was founded to combine microbiome sequencing with ML, and that its PREDICT research data trained models underlying ZOE scores. Its retail gut test uses shotgun metagenomics, though its ML claims also cover its broader personalized-nutrition predictions rather than only the stool-test report. |
| BIOHM Health | Markets the Longevity Gut Score as using “advanced AI,” and trade reporting attributes its aging-related model to machine- and deep-learning methods trained on more than 10 million data points, |
| FeelGut | Claims its sequencing reads are processed through a “machine learning bioinformatics pipeline,” compared with reference libraries and its own data set to produce health scores and food suggestions |
| EZBiome | Says it applied ML and AI to a database of more than 120,000 people to generate a global microbiome health index. |
| Tiny Health | Tiny Health says it uses machine learning with an individual’s sequencing results and survey data to personalize its diet, supplement, and lifestyle recommendations. |
Perplexity also informs me that none have “not publish enough to reproduce its proprietary scoring and recommendations.” Thus the question must be asked, is this marketing hype or validated peer-reviewed science.
Looking at some of the literature
Microbiome ML results can look better than they generalize because the data are sparse, compositional, high-dimensional, and highly affected by batch effects. Differences in DNA extraction, sequencing platform, 16S variable region, taxonomic database, geography, diet, medication exposure, and disease-site recruitment can all be learned by a model instead of the biological signal of interest.
Extract from Machine learning approaches in microbiome research: challenges and best practices [2023]
Which further stated
Microbiome data is often high-dimensional, with more features (microbial genes or taxa) than samples. This can lead to overfitting and poor generalization, especially with small sample sizes. Feature filtering and selection methods are employed to reduce dimensionality, but different methods can yield different results, and correlated features can hinder selection.
The different results from using different methods is a major red flag🚩 for me.
The substantial variability in results produced by different analytical methods is a significant concern. In colloquial terms, there is a risk that artificial-intelligence systems may generate plausible but poorly supported inferences, while analysts may—intentionally or unintentionally—select modeling choices that align with organizational expectations. Highly favorable results can sometimes be obtained through extensive tuning, but the central methodological question remains: do the findings reflect genuine signal in the data, or artifacts introduced through model selection and optimization?
To examine this issue, I conducted a preliminary evaluation of relatively naïve machine-learning models across several collections of microbiome samples, including samples originating from BiomeSight, uBiome, and Ombre. Here, “naïve” refers to simple, out-of-the-box implementations using Microsoft.ML.Data and publicly downloadable data, without extensive feature engineering, hyperparameter optimization, or other model-tuning procedures.
The Core Code in C#
var classifier = new VariableLengthBinaryClassifier();
var data = DataDal.ML_Symptoms(source, sympid);//Array of {condition, double[] }
IReadOnlyList reports =
classifier.Train(data);
var result = classifier.Predict(data[0].Values);
var line = $"{source},{name},Accuracy={reports[0].Accuracy}; ";I evaluated two representations of taxonomic abundance:
- Relative abundance expressed as percentages.
- Quintile-based categories:
- Not detected
- Detected through the 25th percentile
- 25th–50th percentile
- 50th–75th percentile
- Above the 75th percentile
My preregistered practical criterion was straightforward: model accuracy should exceed 0.50. Because a binary classifier can achieve approximately 0.50 accuracy through random prediction under balanced classes, results below this threshold would provide little evidence of useful predictive performance. Across several hundred tested scenarios, the full quintile taxonomic model has associations between 0.429 and 0.447 over 1919 dimensions . Detailed results are provided in the appendix.
Interpretation of the findings
These results require careful qualification. More favorable predictive performance may be achievable when analyses are tightly controlled for population characteristics, sequencing methodology, bioinformatic processing, and other sources of technical and biological variation. However, models developed under such restrictive conditions may generalize only to highly similar populations and to data processed through the same microbiome-analysis pipeline. This limitation is particularly important given the substantial effects that laboratory methods, reference databases, taxonomic classification procedures, and other pipeline choices can have on microbiome results.
The samples used in this evaluation were comparatively heterogeneous: they were uploaded by individuals from diverse locations and backgrounds, although samples within a given retail-laboratory source were processed using the same general pipeline. For potential clinical application, such “real-world” heterogeneity is important, because clinical tools must ultimately perform outside narrowly selected research cohorts. Any published findings should therefore not be interpreted as universal evidence from machine learning in microbiome research; rather, they indicate limited suggestions for the specific datasets, prediction tasks.
Impact of restricting to a Taxonomy Rank
A follow up exploration using quintile-based categories filtered to specific taxonomy ranks resulted in the following accuracy.
- Phylum: 0.73 over 31 dimensions
- Class: 0.73 over 61 dimensions
- Order: 0.73 over 123 dimensions
- Family: 0.64 over 268 dimensions
- Genus: 0.52 over 524 dimensions
- Species: 0.46 over 795 dimensions
In general, as the number of vectors (taxonomies) increases, the reported accuracy decrease.
Metabolite-based analysis
In earlier work, I found that estimated metabolite profiles were more useful symptom predictors than taxonomic abundance alone. These metabolite estimates were derived using functional inferences based on the Kyoto Encyclopedia of Genes and Genomes (KEGG).
A subsequent analysis using metabolite-based features produced substantially better results. In most scenarios, accuracy exceeded 0.50, and the highest observed accuracy was 0.609. This suggests that inferred functional or metabolic characteristics may contain more clinically relevant predictive information than taxonomic composition alone, at least for the outcomes examined.
Over 2293 dimensions, accuracy ranged from 0.56 to 0.618, a much larger spread then above.
Clinical implications
This produces an important practical dilemma:
| Feature type | Primary advantage | Primary limitation |
| Taxonomic profiles | A large literature describes interventions, dietary factors, and substances associated with changes in particular taxa | Taxonomy showed limited predictive performance in these minimally tuned models |
| Estimated metabolites | Better predictive performance in this analysis | Comparatively limited evidence identifies interventions that reliably modify specific inferred metabolites |
At present, I therefore prefer alternative statistical models and taxonomic features for generating practical suggestions, largely because the supporting intervention literature is more extensive. Although retail microbiome-testing companies may indeed use machine-learning methods, I remain cautious about the predictive accuracy and clinical utility achieved by their proprietary models. Claims of machine-learning capability may be commercially attractive, but the relevant question is whether these approaches produce reproducible, clinically meaningful improvements for clients. Peer-reviewed evidence demonstrating such benefit remains the standard needed to support those claims.
At this time of writing, there were 555 users in the last year uploaded to the Microbiome Prescription citizen-science platform. 326 of these users have done two or more uploads in the same year or a 59% repeat rate. This suggests that the suggestions provided were sufficiently beneficial that about 60% of users did a repeat test to get new suggestions.
Recent Comments