In the last two days I was involved in lively twitter discussions about the results of Phase 2 PreSERVE-AMI clinical trial, sponsored by NeoStem. Unfortunately, interpretation of results divided professionals for two camps – (1) trial is failed and (2) trial is successful. Why did it happen? Maybe the truth somewhere in the middle? I’d like to share some of my thoughts and post some questions for discussion here.
Definition of trial failure
Clinical trials can fail for many reasons, not only because investigational product does not work. For example, bad trial design, wrong endpoints, unanticipated slow enrollment and logistical difficulties, sponsor out of money, changing business strategy, termination due to complications and safety issues, termination due to lack of feasibility and technical difficulties – all these and many other reasons could lead (or contribute to) trial failure. In case of NeoStem trial it was quite simple – according company’s press release and trial record on NCT database, primary endpoints were not met. Failure = endpoints not met! It was… until…
Changing of trial record in registry
Company described trial results as “positive”, because some measures were significantly different from placebo. However, those were other than primary endpoints readout measures – serious adverse events (SAEs), major adverse cardiac events (MACE) and survival benefit. In the night of news release, some analysts started to point out that trial is failed because primary endpoints were not met. And the company reacted to it by changing trial record on NCT database! I noticed that record was changed after press release, not any time before. You can see changes before and after Nov. 17, 2014 (night of press release) on this screenshot:
Thanks NCT for archiving all changes! So, now, based on the current record, endpoints are met! Magic! Some questions for discussion:
- How is it legitimate to change trial record as reflection to public reaction on the night of press release? If it’s ok to do, is it a common practice for companies?
- Were “new endpoints” approved by FDA as “acceptable changes” on the middle (or close to the end) of trial? If we assume (I really doubt it) it was, is it a common practice? If we assume NeoStem got FDA blessing to change endpoints (to fit the data) on a middle of the trial, why not update record right away instead of holding it until press release night?
Digging into endpoints
“Initial primary endpoints” of trial, which failed, were myocardial perfusion by SPECT imaging. So, there was no difference in blood perfusion of affected hearts. Was it because of lack of method sensitivity (as company put in press release) or because of other reasons? Maybe CD34+ cells did not impact angiogenesis/ vasculogenesis in any way? Maybe the mechanism of action was different from anticipated? We don’t know for sure.
Now, “new endpoints” – SAE, MACE and mortality were statistically significant. Results for MACE were dose-dependent. One year mortality was less in “CD34+” group compared to placebo. These are good news! Good for patients. These “new endpoints” were born in discussions with FDA in order to move forward:
… which is consistent with U.S. Food and Drug Administration (FDA) guidance that mortality and MACE are the appropriate approvable endpoints to determine efficacy of a cellular therapy for cardiac disease as opposed to imaging endpoints.”
- Is mortality readout at 1 year the best time point? Why not at 3 months or 6 months or 3 years? Are 3 deaths in placebo group versus 0 in “cells” group enough for confident conclusions?
- What was a number of patients in each cell dose group? If we look at only “>20 million cells” group with significant difference in MACE, was number of patients strong enough for confident conclusions? If every 5 million of cells make a difference, was the good positive trend in patients, received 25M, 30M, 35M…. of cells? How much cells were more than 20 – could be 21 or 50 or 100?
Heating up anticipation of results release and investors reaction
NeoStem stock was going up like a rocket few days before press release (+40% in last 2 days of the last week). The company did a good job to heat up anticipation of results. First, a week or so before was news circulation about upcoming release. Second, AHA released abstracts from its annual meeting (where results were reported to public) a week or so before and we were able to read NeoStem’s abstract. There were no any results in abstract. My question is – what was a point of this abstract?
So, investors rewarded the company in anticipation of results, but dumped it in next day of release. It was -29% on Tuesday, November 18 and -9% today.
It seem to me that NeoStem got the same problem as many other public “stem cell companies” in the field – hyping press releases. Let me just tell you how I see “normal” / balanced/ sober/ not hyped press release:
- Please start from endpoints, listed on trial record and say (a) met or (b) not met. There is nothing more important than that. I’d have a respect for the company, which can say “We failed to meet primary endpoints” in press release.
- Please linked to trial record on database, so reader can click on it, compare with press release language and trace possible discrepancies.
- Please avoid use “hype words”. It makes some readers suspicious.
- Press release should be crystal clear, should leave no doubts and no different interpretations.
The NeoStem’s release started from “positive” in title and continued with all good things – SAEs, MACE, mortality, but not “initial primary endpoints” (perfusion by imaging). Important “primary endpoints” came up somewhere in the middle and were “blurred” by reference to FDA, which says “appropriate approvable endpoints are mortality and MACE”. So, don’t worry guys, we don’t need these perfusion data, anyway.
As we can see some results are promising and, I think, company on a good track with it, because:
- they now know what cell dose to pick for future trial
- they now know how to choose endpoints wisely
- they now know how to design the trial correctly in order to expect good results
- they in constant communication with FDA and in agreement with agency on “optimal” endpoints for future
- they probably will not have any problems with FDA in submission for Phase 3 trial…
But, but, but it does not cancel the failure. It does not nullify the “unintended mistakes”, which they made with picking primary “imaging endpoints” (who knew that it is not going to work at the beginning of trial?). It does not nullify “intended mistake” with changing database record and twisting results for public as entirely positive. I think, mistakes and failures should always be acknowledged.
Despite company’s optimism, there will be challenges ahead. For example, manufacturing issues, related to purification of >20M CD34+ cell number from single bone marrow aspirate. As Robert Preti said in release – they are still “refining the methods for collecting and purifying CD34″. So, by the end of Phase 2, the manufacturing process is “not locked”. The other challenge is figure out the mechanism of action and finalize the potency assay. The company positioned mechanism of actions of NBS10 as pro-angiogenic. But lack of any detectable changes in myocardium perfusion (SPECT imaging) does not support this mechanism. We also have to remember that results of Phase 3 must be very very good in order to beat competitors on the market.
I’d like to point out that released results are preliminary. It was nicely highlighted by NeoStem in the title as “initial data”. The final results could be different – in positive or negative way. As of now, we cannot learn a lot of information from press release. We have to wait for publication and learn more about the trial and results.
To conclude: The positive spin of preliminary results of Phase 2 PreSERVE-AMI trial caused unanticipated confusions. Changing of trial record to fit the data in the night of news release is not good practice and will undermine company’s credibility.