Of “Ground-Breaking” and “Breakthrough” and “Blockbuster”: Endpoints Matter

BreakthroughThis week in San Diego the American Association for Cancer Research is holding its annual meeting. When I was an active cancer researcher this was one of the premier meetings that I made sure I attended if at all possible. The meeting itself is high enough profile that many top researchers from around the world wait to announce significant results at this meeting, because they know that the rest of the cancer research world will be watching and listening.

It is not surprising then, that many “breakthroughs” get announced at this meeting. But how many of these are really breakthroughs, and how many of them are just exercises in hyperbole?

I ask this question from personal experience. In the past, many was the time that I myself participated in the preparation of a press release announcing some significant result, usually in a clinical trial but sometimes in basic biomedical research. So I know full well that press releases are meant to hook people, raise the eyebrows of donors in particular, and create a well-deserved excitement for pieces of research that often do not get much public attention. And I know that giving a presentation at such a high-profile meeting guarantees a big audience, and so every researcher wants to put his/her best foot forward when announcing results. This goes doubly for research that has commercial impact, such as a new drug being developed by a company.

One such study exploded onto the airwaves and the Twitterverse this morning.

An investigator team, led by Dr. Richard S. Finn, Associate Professor of Medicine at University of California, Los Angeles announced the detailed results from the so-called PALOMA-1 study, a randomized Phase 2 study of a new drug being developed by Pfizer, Inc called palbociclib that was tested in patients in combination with letrozole. Some patients received only letrozole whereas the others received the new drug plus letrozole. Since this was a phase 2 trial, these are by definition somewhat early results. The trial aimed to look at two different things: progression free survival (PFS) was the main endpoint, but overall survival (OS) was the secondary endpoint.

Because the drug seems to have almost doubled PFS, which you can think of as the time that it would take for the cancer to worsen or to progress clinically, this has naturally made the headlines. In the trial, the drug the average (median) time before measurable disease progression was 20.2 months for those who received palbociclib compared to 10.2 months for the control group. This means that the new drug seems to be able to stall the cancer in its tracks, at least for a time. The longer the time that you can stall the cancer, obviously the better.

The headlines blazed on about how these results are “groundbreaking” and about how this is potentially going to be a “blockbuster” drug.

Andrew Pollack, wrote in the New York Times that this Breast Cancer Drug Shows ‘Groundbreaking’ Results, for example. The lead researcher, Dr. Finn was quoted in the article as saying “the magnitude of benefit we are seeing is not something commonly seen in cancer medicine studies,” and he called the results “quite groundbreaking.”

I don’t know about you, but I am always a bit leery not only of the word “groundbreaking”, but I am especially skeptical when the term is used by the lead investigator him or herself, for obvious reasons of huge self-interest. Noted science writer Ed Yong shares this view as evidenced by a tweet this morning:

Matthew Herper, writing in Forbes that Pfizer Breast Cancer Drug Results “As Impressive As I’ve Ever Seen,” Researcher Says quoted one of the other researchers who showed a similar flair.

“These are as impressive results as I have ever seen. I do not say that lightly.” says Dennis Slamon, a UCLA physician scientist who worked with Pfizer to develop palbociclib and who owns Pfizer stock.

Anyone who follows breast cancer research will instantly recognize that Dennis Slamon is a giant in the field. Despite his obvious stature however, he is no stranger to hyperbole either. One only has to remember a news conference last summer in Toronto where Dr. Slamon and another giant in the field, Dr. Tak Mak, announced news of a cancer “breakthrough” that garnered headlines around the world. The only problem, as André Picard point out in his article in June 2013 Take news of cancer ‘breakthrough’ with a big grain of salt “this seemingly miraculous ‘breakthrough’ drug has not been tested on a single person. The experimental drug CFI-400945 has ‘prevented cancer growth’ in a bunch of mice. The promising results have yet to be published in a peer-reviewed medical journal (though articles have been submitted), let alone replicated by others.”

I am NOT trying to compare the two situations, merely to point out a common modus operandi which is to focus on the positive and ignore that part of the story that isn’t quite as promising.

Pfizer has great reason to focus on the positive, of course. As Deena Beasley of Reuters noted:

The U.S. Food and Drug Administration has granted “breakthrough” status for palbociclib. Pfizer is still discussing a regulatory pathway for the drug and has not decided whether to seek accelerated approval based on Phase 2 trial results, said Mace Rothenberg, chief medical officer for Pfizer’s oncology unit.

Palbociclib is viewed as one of the company’s most important experimental drugs that some analysts believe could eventually claim annual sales of more than $5 billion, if approved by regulators.

So obviously, there is a lot at stake. But that is all the more reason why journalists and reporters need to make sure that reporting is balanced and not over exaggerated. And we have to ensure that the researchers themselves don’t fall prey to the tendency to overstate results

Why do I think that the results are a bit overstated?

First of all let me say that I think this is a very important piece of work. It looks at a new type of drug that is not part of our arsenal at the moment. As such I think this is a very important study as it may lead the way to a whole new avenue of treatment for women with breast cancer. The very fact that in a small study this drug can slow down the progression of breast cancer is itself not a trivial outcome. So, while I think the word “breakthrough” is way too strong, there is no doubt that as a sort of proof-of-principle this drug could lead us to a whole new way of thinking about treatments for breast cancer. For that singular reason this study is important.

But in the end, slowing down the progression of breast cancer, unless it stalls it forever, is not the end goal that we are searching for. Progression free survival is important, but overall survival is what we really need to focus on. If the breast cancer is held at bay for a period of time, but the patient ends up dying just as soon as she would have in the absence of the drug, then there is not much ultimate benefit to her. And this is where I have to say, “whoa, not so fast”.

While it is true that patients showed a very statistically significant improvement in PFS, the OVERALL SURVIVIAL rate, was 37.5 months for the new drug compared to 33.3 months without. Clearly extending someone’s life by 4.2 months is not trivial, especially if you are the patient. But these results were not deemed to be statistically significant and therefore their clinical significance remains to be seen.

To be fair, this is still a phase 2 study, and it is possible that a much bigger improvement in overall survival may show up as patients were followed for a longer time. Indeed that is the hope. And indeed PFS can often be a good marker for OS, so in time we may well have our “blockbuster”: for sure one cannot realistically say that the drug will not make a big difference in overall survival.

All one can say is that the results of this trial do not yet show a significant improvement in overall survival. That is not playing semantics, that is the reality of phase 2 clinical trials.

So whether or not this is truly “groundbreaking” research, it is for sure a very important finding and potentially a very important therapeutic addition to our breast cancer arsenal. Whether or not this proves to be an “blockbuster” for Pfizer remains to be seen.

I really just wish that we as scientists, and reporters, and communicators of research would learn to bring more balance when we report results. I fully understand why people resort to hyperbole, but it sets up false expectations and in the end it is  the credibility of the scientists and the researchers themselves that suffers the most. I think the public is definitely suffering from “breakthrough fatigue” and as important as this study might be, headlines like we saw this morning really don’t help very much.

     
 
If you enjoyed this post, please consider sharing it, leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.
   

Comments

Of “Ground-Breaking” and “Breakthrough” and “Blockbuster”: Endpoints Matter — 2 Comments

  1. The New York Times has today changed the headline from “..GROUNDBREAKING…” to:
    “… guarded optimism…after PROMISING results”