Is Ocrevus worth trying for PPMS? There’s no simple answer. This is the first of a series of blog posts to help you decide. Before we dig into the pros and cons, you need to be familiar with a few key concepts of drug trials. I call them the Five Ps, and they apply not just to Ocrevus, but any drug. Let’s get started:
In a clinical trial, a group of patients are given a drug, then the its effectiveness is measured. In some trials, there’s also a second group that receives a placebo. This appears to be the same drug, but actually does not contain any active ingredients. This is called a placebo control.
One example is all it should take to convince you of the importance of placebo controls: 49% of patients in the in the Phase 3 (see below) Ocrevus PPMS trial suffered respiratory tract infections. Was this due to the drug? With no similar placebo group, we have nothing to compare it to. Fortunately, this study had a placebo control, and 43% of those had respiratory tract infections. So we know most of the infections can be attributed to other causes.
Well designed studies are randomized, which means the drug company doesn’t get to choose which patient gets the placebo, and which gets their drug. As the name implies, patients are assigned to groups at random.
Usually, more people in a study get the real drug than the placebo, since you can get the information you need from the placebo group with a smaller population. Also, drug trials may switch groups after a period time, giving the placebo group the real drug, and vice versa, to determine if earlier treatment is more effective.
Some drug trials don’t have control groups, since the drug’s side effects make it obvious whether or not you’re getting the real thing. A chemotherapy treatment, for example.
A “p-value” is a number that tells us how effective a drug is. Here’s an easy to understand example:
You and your buddy Bob play poker with some less experienced friends every day for a month. Then you each add up your winnings. Bob has won $62 per day on average, and you’ve won $57.
Is Bob a better poker player than you? Or did he just get lucky, by being dealt the right cards?
We can answer that by calculating a p-value – the odds Bob’s larger winnings are due to chance. For example, given these 30 days of poker winnings (in dollars):
The p-value is about .025. That means there’s only a 2.5% chance Bob’s higher winnings are due to luck alone. While we can’t say with 100% certainty that Bob is the better poker player, we can be highly confident that he is.
This is exactly the question we ask about the results of Ocrevus versus placebo – is it really an effective drug, or did it just get lucky in single study? What p-value would you need to consider a drug effective?
The requirement for drug approval by the FDA is a p-value of .05 – and it’s a pretty strict standard. In fact, in 2006, the FDA recommended against approving the prostate cancer drug Provenge because the p-value of one of test was .052.
Use common sense when looking a p-values. For example, a drug for a disease you have fails FDA approval with a p-value of .2. The drug is free, and has no side effects. Would you take it? Probably, because there’s an 80% chance it is indeed effective. Would you take it if it cost half your annual income, and came with serious side effects? Then that 20% might become important.
When a drug company thinks a drug shows promise, it usually tests it in cell cultures or animals first (monkeys in the case of Ocrevus). If the drug is shown to be sufficiently safe and effective there, four types of human trials may be initiated:
- Phase 1. The goal is here is safety. This usually involves a small group of patients who are given varying dosages of the drug. There is no placebo group.
- Phase 2. A larger group than Phase 1. Safety, effectiveness, and dosage are tested. Usually no placebo group.
- Phase 3. A larger group than Phase 2. A specific primary endpoint (see below) is defined. Placebo controlled. This is the key phase for FDA approval.
- Phase 4. After regulatory approval. A study to determine the long-term effects of the drug.
Drug companies like to give their trials fun and distinctive names, usually written in all caps. For example, the four Phase 3 Ocrevus trials for rheumatoid arthritis (RA) were called STAGE, SCRIPT, FILM, and FEATURE. The Phase 3 PPMS study was called ORATORIO.
Sometimes these studies are combined. For example, two studies were carried out to determine the effectiveness of Ocrevus for RA. The results:
- Combined Phase 1 and 2 study: 237 patients were treated with varying dosages ranging from 10 to 1,000 mg per infusion. The results were promising, and optimal safety and effectiveness were achieved at the 200 mg dosage.
- Phase 3 study: 1,932 patients. Primary endpoint: 20% improvement in a composite test of RA symptoms. Result: primary endpoint met, p value < .001. Unfortunately, due to a number of opportunistic infections, Genentech suspended development of RA with Ocrevus.
A drug may have a number of effects on a disease. Ocrevus is a perfect example: it might reduce brain lesion load, improve walking times, and increase manual dexterity. At the beginning of a trial, the drug company states the specific benefits it expects the drug to have. These are called trial “endpoints”.
A clinical trial can have many endpoints, but it usually has just one primary endpoint. This is the critical benefit the drug must show for FDA approval. Why just one? It’s a problem of p-values. Here’s how the FDA describes it:
“In a clinical trial with a single endpoint tested at p-value = 0.05, the probability of finding a difference between the treatment group and a control group by chance alone is at most 0.05 (a 5 percent chance).
By contrast, if there are two independent endpoints, each tested at p-value = 0.05, and if success on either endpoint by itself would lead to a conclusion of a drug effect, there is a multiplicity problem. For each endpoint individually, there is at most a 5 percent chance of finding a treatment effect when there is no effect on the endpoint, and the chance of erroneously finding a treatment effect on at least one of the endpoints (a false positive finding) is about 10 percent.”FDA, Multiple Endpoints in Clinical Trials Guidance for Industry
In other words, defining multiple endpoints weakens the power of any single endpoint, and increases the chance your drug may not be labelled by the FDA as effective for any endpoint at all.
To get around this problem, drug makers can declare “secondary endpoints” to support the success of the primary endpoint. Again, though, the drug almost certainly won’t be approved unless the primary endpoint is met.
It’s worth paying attention to endpoints because drug companies can manipulate them. For example, assume a drug company knows a drug is dangerous, but only after a patient has been taking it for a year. They could define a short study endpoint – say, 12 weeks, and ensure the drug appears safe in the study.
For a clinical trial, you need a bunch of people, and it’s important to pay attention to how these are chosen. There are three main things to watch for:
- Quantity. The more subjects in the study, the greater the “power”, or statistical effectiveness, of the results. This usually isn’t a problem in Phase 3 studies, which have sufficiently large populations.
- Characteristics. Imagine you’re 60 years old, and have had a rare type of cancer for 10 years. An apparently safe and effective drug is approved for your disease. You’re initially excited, then you look at the trial population and notice something startling – the average age of the study patient is 30, and they’ve only had the disease for an average of two years. What’s safe and effective for this group may not be for you.
- Dropouts. In the Ocrevus Phase 3 PPMS study, the primary endpoint was “time to confirmed disease progression”. Here, “confirmed” means the patient was tested again, 12 weeks after the initial progression, to confirm it.
Now imagine a patient who tests positive for progression, but drops out before it can be confirmed 12 weeks later. How to count him?
Or imagine a patient dies midway through the study. Was this due to the drug? How do you count him from a safety standpoint?
In cases where a drug is found to be highly effective, dropouts and patient characteristics are less important. But in a marginal product like Ocrevus, they become major issues, as we’ll see in future blog posts.