Guidelines Fall Short of Best Scientific Practices


USDA officials directing the Guidelines stated earlier this year that their scientific reviews would follow the “GRADE” system for evaluation of the science. GRADE is considered one of the top systems in the world for producing reliable scientific reviews and guidelines. However, in a recent public comment to the USDA, GRADE co-founder, Dr. Gordon Guyatt, expressed strong concerns that the USDA’s decision to “modify” GRADE would lead to Guidelines that are “unlikely to be trustworthy.” A principal problem, wrote Guyatt, is that the USDA has no methodology to distinguish between high- and low-quality evidence. “This distinction between high- and low-quality evidence lies at the core of any rigorous evaluation of science and is at the heart of the GRADE methodology,” stated Guyatt. A Distinguished Professor in the Department of Health Research Methods at McMaster University, he urged the USDA not even to use the word GRADE, “because doing so would give the appearance of rigor where it did not exist.”

Other Serious Questions About USDA’s Science Reviews

Another recent public comment to USDA, by Dr. Bradley Johnston, director of an independent group of international researchers who are leaders in high quality systematic reviews of nutrition science, stated that the USDA’s “proposed methodology…deviate[s] significantly from basic scientific precepts in a number of important ways. Taken together, these deviations…from international standards for systematic review methodology will result in a non-systematic approach that would seriously undermine the reliability of these reviews.”

Johnston, also a GRADE expert, stated that among other things, USDA’s decision to rely on its previous systematic reviews makes little sense, since USDA was cautioned by the National Academies of Sciences, Engineering, and Medicine (NASEM) that these earlier reviews used non-systematic approaches. “Thus, relying on these previous reviews would mean incorporating evidence that…is of questionable reliability,” wrote Johnston. He also questioned USDA’s use of “hand-searches” for science when the vast majority of scientific literature is online. Johnston concludes, “…based on the available documentation from CNPP [USDA], the proposed 2020-2025 U.S Dietary Guidelines for Americans will be fundamentally lacking in scientific rigor and will not comply with the upgrades in scientific methodology that NASEM has called for.”