Some thoughts on results of NeoStem cardiac cell therapy trial

by Alexey Bersenev on November 19, 2014 · 2 comments

in business

In the last two days I was involved in lively twitter discussions about the results of Phase 2 PreSERVE-AMI clinical trial, sponsored by NeoStem. Unfortunately, interpretation of results divided professionals for two camps – (1) trial is failed and (2) trial is successful. Why did it happen? Maybe the truth somewhere in the middle? I’d like to share some of my thoughts and post some questions for discussion here.

PRIME-XV-Expansion-Banner-550x80

Definition of trial failure
Clinical trials can fail for many reasons, not only because investigational product does not work. For example, bad trial design, wrong endpoints, unanticipated slow enrollment and logistical difficulties, sponsor out of money, changing business strategy, termination due to complications and safety issues, termination due to lack of feasibility and technical difficulties – all these and many other reasons could lead (or contribute to) trial failure. In case of NeoStem trial it was quite simple – according company’s press release and trial record on NCT database, primary endpoints were not met. Failure = endpoints not met! It was… until…

Changing of trial record in registry
Company described trial results as “positive”, because some measures were significantly different from placebo. However, those were other than primary endpoints readout measures – serious adverse events (SAEs), major adverse cardiac events (MACE) and survival benefit. In the night of news release, some analysts started to point out that trial is failed because primary endpoints were not met. And the company reacted to it by changing trial record on NCT database! I noticed that record was changed after press release, not any time before. You can see changes before and after Nov. 17, 2014 (night of press release) on this screenshot:

NBS
NBS1
Thanks NCT for archiving all changes! So, now, based on the current record, endpoints are met! Magic! Some questions for discussion:

  1. How is it legitimate to change trial record as reflection to public reaction on the night of press release? If it’s ok to do, is it a common practice for companies?
  2. Were “new endpoints” approved by FDA as “acceptable changes” on the middle (or close to the end) of trial? If we assume (I really doubt it) it was, is it a common practice? If we assume NeoStem got FDA blessing to change endpoints (to fit the data) on a middle of the trial, why not update record right away instead of holding it until press release night?

Digging into endpoints
“Initial primary endpoints” of trial, which failed, were myocardial perfusion by SPECT imaging. So, there was no difference in blood perfusion of affected hearts. Was it because of lack of method sensitivity (as company put in press release) or because of other reasons? Maybe CD34+ cells did not impact angiogenesis/ vasculogenesis in any way? Maybe the mechanism of action was different from anticipated? We don’t know for sure.
Now, “new endpoints” – SAE, MACE and mortality were statistically significant. Results for MACE were dose-dependent. One year mortality was less in “CD34+” group compared to placebo. These are good news! Good for patients. These “new endpoints” were born in discussions with FDA in order to move forward:

… which is consistent with U.S. Food and Drug Administration (FDA) guidance that mortality and MACE are the appropriate approvable endpoints to determine efficacy of a cellular therapy for cardiac disease as opposed to imaging endpoints.”

Some questions:

  1. Is mortality readout at 1 year the best time point? Why not at 3 months or 6 months or 3 years? Are 3 deaths in placebo group versus 0 in “cells” group enough for confident conclusions?
  2. What was a number of patients in each cell dose group? If we look at only “>20 million cells” group with significant difference in MACE, was number of patients strong enough for confident conclusions? If every 5 million of cells make a difference, was the good positive trend in patients, received 25M, 30M, 35M…. of cells? How much cells were more than 20 – could be 21 or 50 or 100?

Heating up anticipation of results release and investors reaction
NeoStem stock was going up like a rocket few days before press release (+40% in last 2 days of the last week). The company did a good job to heat up anticipation of results. First, a week or so before was news circulation about upcoming release. Second, AHA released abstracts from its annual meeting (where results were reported to public) a week or so before and we were able to read NeoStem’s abstract. There were no any results in abstract. My question is – what was a point of this abstract?
So, investors rewarded the company in anticipation of results, but dumped it in next day of release. It was -29% on Tuesday, November 18 and -9% today.

Positive spin
It seem to me that NeoStem got the same problem as many other public “stem cell companies” in the field – hyping press releases. Let me just tell you how I see “normal” / balanced/ sober/ not hyped press release:

  1. Please start from endpoints, listed on trial record and say (a) met or (b) not met. There is nothing more important than that. I’d have a respect for the company, which can say “We failed to meet primary endpoints” in press release.
  2. Please linked to trial record on database, so reader can click on it, compare with press release language and trace possible discrepancies.
  3. Please avoid use “hype words”. It makes some readers suspicious.
  4. Press release should be crystal clear, should leave no doubts and no different interpretations.

The NeoStem’s release started from “positive” in title and continued with all good things – SAEs, MACE, mortality, but not “initial primary endpoints” (perfusion by imaging). Important “primary endpoints” came up somewhere in the middle and were “blurred” by reference to FDA, which says “appropriate approvable endpoints are mortality and MACE”. So, don’t worry guys, we don’t need these perfusion data, anyway.

The future
As we can see some results are promising and, I think, company on a good track with it, because:

  • they now know what cell dose to pick for future trial
  • they now know how to choose endpoints wisely
  • they now know how to design the trial correctly in order to expect good results
  • they in constant communication with FDA and in agreement with agency on “optimal” endpoints for future
  • they probably will not have any problems with FDA in submission for Phase 3 trial…

But, but, but it does not cancel the failure. It does not nullify the “unintended mistakes”, which they made with picking primary “imaging endpoints” (who knew that it is not going to work at the beginning of trial?). It does not nullify “intended mistake” with changing database record and twisting results for public as entirely positive. I think, mistakes and failures should always be acknowledged.

Despite company’s optimism, there will be challenges ahead. For example, manufacturing issues, related to purification of >20M CD34+ cell number from single bone marrow aspirate. As Robert Preti said in release – they are still “refining the methods for collecting and purifying CD34”. So, by the end of Phase 2, the manufacturing process is “not locked”. The other challenge is figure out the mechanism of action and finalize the potency assay. The company positioned mechanism of actions of NBS10 as pro-angiogenic. But lack of any detectable changes in myocardium perfusion (SPECT imaging) does not support this mechanism. We also have to remember that results of Phase 3 must be very very good in order to beat competitors on the market.

I’d like to point out that released results are preliminary. It was nicely highlighted by NeoStem in the title as “initial data”. The final results could be different – in positive or negative way. As of now, we cannot learn a lot of information from press release. We have to wait for publication and learn more about the trial and results.

To conclude: The positive spin of preliminary results of Phase 2 PreSERVE-AMI trial caused unanticipated confusions. Changing of trial record to fit the data in the night of news release is not good practice and will undermine company’s credibility.

{ 2 comments… read them below or add one }

Bruce Michael November 20, 2014 at 12:46 am

Very nice write-up. I agree with many of the points, we must learn from the mistakes and not forget them. There were many things to be learned from this trial, being one of the first and largest cardiac cell therapy trials in the US to date so it is definitely paving the way for future endeavors. From a compliance and regulatory perspective it was shifty and they nearly got through just as the door was closing which is worth shaking your head at in a way. From a science perspective it’s good to see when you apply a proven safe therapy to a larger number of patients that your data starts to stand out more and you can now monitor its results over time to truly see it’s activity. Overall, we are at the beginning of all of this and it’s tough always being the first person to step forward and take the initiative and accept the challenge but it makes it easier for others to follow to lead so we can all hopefully work together in providing patients with more effective and robust therapies.

Reply

Alexey Bersenev December 3, 2014 at 11:46 pm

Because of new NeoStem press release and some data from the trial, I’d like to update this post in comment.
1. Few days after trial results announcement, NeoStem released second “clarifying” PR. And the reason for clarification was:

we are extremely disappointed, in the face of exciting results, that shareholder value is being diminished by pervasive inaccurate or incomplete statements about the study,” said Dr. Robin Smith, Chairman and CEO of NeoStem.

Company said that they filed new “endpoints” to FDA as amendment to clinical protocol on July 14, 2014. However they failed to address the following important questions:
When was amendment approved by FDA?
– Why company chose to disclose this change after release of trial results?

One important point from second PR is

the Company’s decision to expand the primary endpoints was done prior to the Contract Research Organization (CRO), which holds all data related to the study, locking the database and performing the initial analysis.

You can read about the best practices for changing trial endpoints here.

It took 2 press releases, 3 blog posts and 1-2 conference calls/ webcasts to report PreSERVE-AMI data, and, after all those, we still left with questions. How credible and wise is it? Yesterday, NeoStem’s CEO wrote in the blog:

We believe the market response does not correlate with the views expressed by the thoughts leaders in this field and as such we will continue to educate the market.

Oh, common NeoStem! This is childish! What do you guys expect after being not transparent with your investors and calling them “not educated”? You made mistake and investors fairly punish you (dropping stock for 30%)! You earn it! You still don’t know how to write press releases to gain investor’s trust. The lesson you have to learn – not only data drive the market, but also honesty and transparency!

2. Cell dose and number of patients per group.
This slide explains the differences:
https://twitter.com/johnnykabuki/status/535706954908180480
Let’s look at number of patients: <14M = 47, >14M = 31, >20M = 15. To my question about feasibility of CD34+ isolation >20M – only 16%! What are chances that company will able to get >20M CD34+ cells in all patients in Phase 3?

One more thing – since there is no evidence for perfusion improvement, the question of mechanism of action and potency assay is still open.

If I miss something from webcasts and conference calls, please educate me in comments.

Reply

Leave a Comment

Previous post:

Next post: