Sarah Kliff reports on a new study showing that portion size matters. Here’s the nickel version: If you offer kids a bowl full of cookies, they eat more than if you offer them a bowl full of smaller cookies.
Well, that sounded interesting. But I was curious about the details. So I found the original dissertation, which is ungated, and browsed through it. First there was this:
Recruitment was performed in an elementary school attended by one of the experimenter’s children. School director and legal guardians (i.e. mother/father) gave all written informed consent and both 1st and [] 6th grade classes participated voluntarily. The experimental protocol was approved by the Ethical Committee of the Faculty of Psychological Sciences of the Université Libre de Bruxelles.
OK, that has nothing to do with the research. It just cracks me up. They had to get IRB approval to feed some kids whole cookies and other kids half-size cookies! This is modern science, red in tooth and claw.
Moving on, there’s a detailed description of the cookies (several layers of wafers filled with milk chocolate topping) and the number of kids (85 total, about half of them girls). The experiment took place at 2:45 pm (the children’s “usual snack time”). Here’s the experimental design:
Kids were called up in alphabetical order, reported pre-study hunger (4-point scale labeled ‘not at all’, ‘a little’, ‘fairly’ and ‘a lot’) and were randomly assigned to a room and table. Children were seated in front of square tables. Individual plates were used in order to stick to maximize external validity, as all children bring their own afternoon snack. For first graders, nine tables were filled with four and one with three children. For sixth graders, ten
tables were filled with four and two with three children. Children were told they could eat as much or as little as desired and were informed they would be given a refill if they liked to. They were allowed to talk but not to share their food. Experimenters ensured that the foods were not shared and if not consumed, were left on the table.
Let’s just stop right here. There’s a grand total of 85 kids in this experiment, about 20 in each experimental group (half in each grade level, half of each getting whole cookies and half getting small cookies). It was, apparently, run once. It was preceded by having the kids report their “pre-study hunger.” Kids were trooped over to a table and given instructions so specific that even a six-year-old would know something was up. Eight kids were excluded from the analysis due to an unexplained algorithm based on “presence of food allergies, overweight, weight problems, dieting behavior, food intake control in order to gain / lose weight and not hungry.” On the remaining data, “One-sample Kolmogorov-Smirnov was used to examine normality of data distributions and chi-square tests to analyze the independence of the sampling distributions for the habit to eat cookies.” The study apparently made no effort to see if reduced cookie eating had any effect on total caloric intake for the rest of the day. Nor did it follow up to see if the children’s behavior remained the same if they were offered the different cookie choices repeatedly over a period of weeks or months.
Look: portion control is probably important. No argument there. But this study is ridiculous. I have no idea how stuff like this gets published. The small sample size and one-time nature of the experiment should be enough to get it laughed out of town immediately, and the rest of the problems are just icing on the cake (so to speak). Unless someone can convince me that I’m being unfair here, I’d say that on the “Is it science or is it bullshit” scale, this one qualifies pretty firmly as bullshit.
UPDATE: For what it’s worth, no one in comments has convinced that this study is non-ridiculous. After reading a few of the defenses, however, I’m starting to have some doubts about the modern scientific enterprise.
