> Methionine is an amino acid. Research shows low methionine diets increase lifespan in animals. Some recent research: http://ift.tt/1xqWHir _sdtp=
This is not a scholarly way to do citations. Citations should be to specific articles. That will present clear targets for criticism: if the specific articles cited are refuted, then the material citing them would need to be revised or rejected.
Linking to search engine results doesn’t work because search results change over time, and results vary based on the searcher’s IP address and browser cookies. There is no way for me to reliably know which articles were found in the original search, and this situation will get much worse over time.
This makes criticism problematic because someone might read (and perhaps criticize) an article that wasn’t actually intended as a source. It’s important to be clear and specific about what one’s sources are.
This is relevant to the GRG mission because doing scholarship correctly is necessary for research to be effective.
Also note the PDF leans heavily on the cites. There are significant points for which it presents no argument, only relying on cites/external information. Such as low met diets being good. The only arguments for that are via citing, so the cites have to be top notch (but aren’t).
I went ahead and took a look at the top search result. My comments here are intended to be illustrative – many of the same ideas could be applied to other papers:
> Methionine restriction increases blood glutathione and longevity in F344 rats
This paper has many large flaws, both intrinsically and also for use as a source for “Practical Methionine Reduction in Humans”. I present a list including both types of issues.
This was published in 1995, it is not “recent research” as was claimed when providing the source.
– Will It Work In Humans?
Does the same thing work for humans? This source isn’t about that. The “Practical Methionine Reduction in Humans” PDF should either explain reasoning that it would work in humans, or raise the issue and then give a specific source which contains reasoning about it working with humans.
– Mischaracterizing The Research
Now consider the claim quoted above that, “Research shows low methionine diets increase lifespan in animals.”
Does this research show that? No. At best it shows that low methionine diets are *correlated with* increased lifespan (and correlated with increased blood glutathione). That’s different. What they did is feed different rats different diets and then compare results, in order to establish correlations between diet and some measured results.
– Not Reproducible
Is this research reproducible? No. It doesn’t provide adequate information to reproduce it. For example, it doesn’t specify enough details about the cages – e.g. whether there were wheels. To reasonably attempt to reproduce this results (and it should be reproduced, among other things, before humans change their diets) would requiring knowing some details of what they did which are not provided.
Note these details also matter to things other than reproducing the results. For example, if the rats had no exercise, it could be the observed correlation only happens with creatures that don’t exercise. Maybe humans have to avoid exercise for the dietary changes to work. Or maybe the opposite: the rats had wheels and it only works if you do exercise. You simply don’t know when dealing with inadequately controlled or reported experiments dealing with correlations.
– Inability To Count
In the “MATERIALS AND METHODS” section it says animals were housed in groups of 5, but it also says there were 16 control rats. How is this possible? This is a HUGE problem. What is going on? They miscounted their rats or lied about using a 5-per-cage method, or they mixed control and non-control rats in the same cages. This shows a level of extreme carelessness that means all results of the paper basically need to be ignored. At best you could think maybe they were onto something and try replicating it, but you can’t trust the conclusions of work like this.
– Inadequate Controls
As recently discussed at the GRG forum, it’s important to have calorie restricted control rats because if you change anything (e.g. feed rats something different) they might eat less as a result and therefore get calorie restriction benefits. But this experiment says it gave all the rats unlimited food. (BTW if you do have calorie restricted controls, how much do you restrict their calories? That seems problematic too.)
– No Sources Of Error Section
The paper doesn’t have a sources of error section. That’s very unscientific. You have to carefully consider all the ways your results could be false and share them along with your results.
– Unclear Statistical Methods
> Data are expressed as means ± SEM. Differences between groups were considered significant at P < 0.05 by Student’s t test or by ANOVA with Scheffe’s posthoc test.
Why are there two different statistical methods? Which was used when, and why? This sounds like a potential source of error: maybe they used whichever one would give them a bigger result. It’s very important to decide on statistical methods before doing the experiment and stick to them, or else the research is invalidated.
– Correlation vs. Causation and Explanation
When talking about reducing GSH, most of the reasoning given relates to correlations rather than causes or explanations. And the work itself focuses on correlation – which it then falsely presents as causation in the title.
Causes and explanations of what’s going on and how things work are what’s important. Correlations should be used in order to learn about causes and explanations, rather than being the focus. Each correlation should be presented along with clear acknowledgement that it’s a correlation, and a statement about what useful thing we can learn from it (regarding causes or explanations of what’s going on). This isn’t done.
They say their result is remarkable: they restricted Met, the only precursor to GSH the rats had, and then GSH levels were *higher*. That’s interesting, but note it’s a correlation. Then what do they do? Say this correlation “clearly suggest[s]” one particular conclusion, even though it’s logically compatible with many different explanations of the causation. This is very sloppy and an unacceptable way of trying to figure things out. To understand a correlation like this, you have to critically consider many hypotheses about what’s causing it, not just present one and falsely claim the correlation “clearly suggest[s]” it.
Also note given the result is remarkable that met restriction *raises* GSH, we should not be assuming this remarkable result will apply in all animals and humans with all kinds of other factors different. We should be extra careful with it.
– Scientific Method
The authors don’t understand the scientific method. The basic purpose of scientific experiments is to rule things out, but they present it as if they were positively establishing things. (For information on scientific methodology, see the books of Karl Popper and David Deutsch, such as _Conjectures and Refutations_ and _The Beginning of Infinity_.)
A good thing to consider when looking at science is what hypotheses they are trying to test, which may be ruled out by the experiment. What you need in order to learn something is two (or more) hypotheses which make contradictory predictions. Then you do an experiment and at least one is ruled out.
This research isn’t done using that approach to scientific testing. So what good is it?