MathBench > Statistics

Normal Distributions and the Scientific Method

What makes a Good Procedure?

Here's the scenario: You have been hired by the new online mega-petstore G.O.P. (Guaranteed Overnight Pets) to formulate a new fish food. Their current fish food produces rather slow growth, requiring too many days to rear fish to an acceptable weight for being sent through the mail.

You've done the hard work -- tested various kinds of insect larvae, nematodes, and algae, you have created (you believe) the perfect formula, named it "Fish2Whale", and sent it off. You are waiting for your paycheck when instead you receive a rather terse note:

Edgar, the mascotTo Whom It May Concern:

Please be advised that we do not intend to make any payment for your formula, "Fish2Whale". We tried it out on Edgar, the company mascot, and he didn't seem to get very big.

Sincerely,

G.O.P.

You might be excused for feeling a bit put out at this point. Management at G.O.P. has not followed the scientific method when they tested your formula. Below I list several statements about fishfood. Pick out the one(s) which describe a satisfactory test of your fishfood. (This is not the same as picking the statements that you think are the most likely explanations. All we care about is that the idea is stated in a way that is testable.)

 

Good scientific procedure?

Answer Statement
If you feed a group of fish with Fish2Whale and they get to be at least 10 cm long, that proves that Fish2Whale works. not a good procedure: there is no group to compare the Fish2Whale fish with.
You should write back and tell the company that Fish2Whale is healthy for fish and will make their scales shine. This is not a scientific procedure, this is a commercial! Fish2Whale is healthy compared to what? And how will you compare their effects?
You should feed one fish Fish2Whale and one fish normal food, and see which fish gets bigger. it's good that you are comparing your treatment to a control, but what if the 'treatment' fish (the one you're feeding Fish2Whale to) happens to be sickly?
You should write back and tell them that it's stupid to use the old food when they've already bought the new food. "Stupid" is a value judgement, not a testable proposition. This is not even a commercial, it's just career suicide!
You should feed normal fish food to one group of fish (the control group) and Fish2Whale to another group of fish (the treatment group) and see which group does better. This is pretty close. You have a comparison and replication, but you don't have a way to measure which group is "doing better" (length? weight? number of surviving offspring? scale shininess?)
You should feed normal fish food to one group of fish (the control group) and Fish2Whale to another group of fish (the treatment group) and see which group grows longer in length. This is a sound scientific procedure. You have a way to measure success, something to compare your success to, and replication to ensure that random effects don't screw up your experiment.


Answers:
no, no, no, no, no, yes

Once again, make sure that when you design an experiment, you think in terms of MEASURE-COMPARE-REPLICATE. And when you are analyzing a statement above, look to see how the authors used each of these elements.