Hello, I’m Dr. Jeff Kingsley and welcome to another edition of Riding in Cars With Researchers. Today I want to talk about statistics. This conversation has come up so often over the last few weeks for me with research sites, with Sponsors, and with CROs. These are very intelligent people in the industry and I continually get an AHA moment when people recognize the breakdown of statistics at the research site. Statistics work with very large data sets. If you think about it, if you’ve ever done any work with statistics or if you think about how we design our research protocols for that matter, you have to have a certain number of patients in order to have enough power to be able to say “A caused B”. Statistics work better the larger the dataset. The smaller the dataset, the more inaccurate things become.

Sick Patients

Now, let’s make that very simple. Flip a quarter. You can see a quarter – it’s a head, it’s a tail. You know that if it’s a normal quarter, if I flip it enough times, it’s going to be 50% heads, 50% tails. If I flip it just a few times, say four, I can get four tails in a row and that’s not terribly uncommon. I’ve run this exercise in Excel if you want to do it. There are simple formulas you can put into Excel for random number generation and so, you could run a series and then see how the series gives you accuracy the larger the number of rows you use, and gives you inaccuracy the shorter the number of rows that you use. Now, why does any of this matter? Well, because if you’re doing a Phase 3 trial and you’re looking to put 3000 patients in, it’s very possible that over the course of your 3000 patients you have a 50% screen failure rate. It’s also very possible that a very competent site, doing all of the right things, has a 100% screen failure ratio because they’ve only put in 4 patients and all 4 were screen failures. They’ve done nothing wrong as that’s perfectly normal random behavior. That site had 4 patients who came in and one patient was a little anemic and that triggered an exclusion criteria, and another patient just got a diagnosis of a pulmonary nodule suspicious for cancer two weeks ago and needs to be worked up, so that patient has screen-failed for yet another exclusion criteria, et cetera. That is the reality of statistics on a small n (a small sample size), and that’s normal at the site side. Now if you’re a site, the sicker your patient population is, the more this will affect you. The sicker your patient population is on a small n (a small sample size), the greater the likelihood that you’ve got patients who are triggering all of these various inclusion and exclusion criteria.

Healthy Patients

The healthier your patient population, the more this is muted. If you’re doing vaccine research with young, healthy people, even on a small sample size, you should have a very small screen failure ratio. And on a small sample size, you should still have very predictable enrollment, randomization, and revenue. The sicker your patient population, the more volatility you will have on enrollment, randomization, and revenue. And it’s not that you’re doing anything wrong as it’s normal probability on a small sample size. So if you’re a site, you need to recognize the sources of your volatility to your enrollment predictability and your revenue predictability, and then decide what things that you want to do to help buffer against that, to hedge against that.