Are people really ok with this practice of researchers talking about their discoveries without having shared the methods and data (I don't care so much about the review aspect)? I'm certainly not, but apparently many think this is acceptable for some reason because I don't see any pushback.
I haven't seen the presentation about the study (a related news briefing is here [1]) so I don't know whether you're correct to say that the researchers haven't shared their methods and data. They seem quite happy to share technical details at the news briefing.
However, regardless of that, I still think it is a good thing if researchers talk about their findings at an early stage, especially if the findings are as dramatic and positive as these. If a treatment is available and you have patients out there dying with no other options, then it is just wrong to hide that away.
In my opinion we treat these things far too cautiously. It is not optimal for the individual and it is not optimal for society. By sharing /and/ opening up options like this to terminally ill patients we can (a) save individual lives and (b) speed up development and refinement of new treatments to benefit everyone.
The problem is that the incentives we have in place treat a cancer death as acceptable, while a treatment-related death is unacceptable for researchers, doctors and drug companies (unless they are shielded by layers of clinical-trial bureaucracy or years of standard practice). The result is that the course of treatment doesn't properly weigh the different sources of risk. Whether the goal is quality-adjusted life years for an individual patient or faster development of effective and safe treatments, the current system is way too conservative.
[1] http://www.eurekalert.org/aaasnewsroom/2016/webcast/?b=38
1) There will be some studies where a measurement was higher in the treatment group compared to controls, but it is not 100% clear what combination of things is actually being measured.
2) There will be other studies you hear about that got different results, but didn't get published because publishing "negative results" is discouraged and bad for careers.
3) The criteria for publishing a positive result is a p-value below an arbitrary threshold, and most people researching medical treatments really do not understand the meaning of a p-value to begin with. There are all sorts of myths like it is the probability the treatment didn't work, etc. So (at least the implementation of) the basic criteria used to judge success is fatally flawed.
It is quite scary and I don't see anything I can do about it but make people aware there is a problem here.
From a technology point of view, I think we are already well past "no one.. really knowing what is going on." Take the current study. Past researchers had identified the CD19 antigen found (often exclusively) on B cells, and others had found that autologous T-cells could persist in a patient for many years, potentially providing a durable response. The current researchers built on this knowledge to design and engineer T-cells that could wipe out a patient's B cells.
So it's a very planned, targeted approach, more akin to engineering than the old approach of testing thousands of random molecules as potential treatments. When the targeted approach works, and especially when it has a large and consistent effect on patients as in this case, then you can be very confident at an early stage.
This needs to be recognized and medical practice needs to change as a result. We're entering an era of incredibly detailed knowledge and precision tools for research and development. The pace of development will continue to increase and treatments will become more patient-specific as individual targets are identified.
You could consider the large, slow, blunt clinical trials as being like optimization for a particular treatment. When new drugs are rare or not well understood it makes sense to spend time to optimize their use. But when new drugs come frequently and are well targeted this premature optimization costs lives.
I didn't watch the video, so I'm not sure where this info is coming from, but providing such references is one reason why they should write a paper up before talking to the media.
Also, I am fairly certain if we do follow up on these claims (I know nothing about them specifically) we will find the actual evidence is much more ambiguous, there has been no direct replication attempted, the references do not actually contain the evidence claimed, etc.
Another approach is to investigate further using the tools available: look up the authors' past work on Google Scholar, check clinicaltrials.gov, look for third party opinions, even ask the authors directly. I'd argue that the scientists and journalists at the AAAS conference where this was presented are capable of this second approach, and so are many participants on HN.
It is too time consuming to research a claim indirectly like that. That is what you do after reading the paper which contains (usually most of) the required information and (usually a biased sample of) citations to the other research that were found relevant.
Reading the paper is only a first filtering step to evaluating a claim, a conference presentation simply doesn't merit that effort because it contains far too little information. A conference presentation is more like "hey, check out what I'm working on".
Now, that doesn't mean people don't try to do this, it is one of the primary uses for misinterpretation of p-values.
Right now the strategy is to kick the can down the road until there is a solution: traditional chemo, then the blunt immunotherapy of ipilimumab + nivolumab/pembrolizumab.
Also (with oncologist's knowledge) adding in very speculative but reasonably safe extras like a COX-2 inhibitor, anti-histamine, and melatonin. (Certainly not the well-targeted approach I talk about in my other comment, unfortunately!)
I hope that you're able to find something that works out.
This technique is actually fairly well known and other groups are running similar trials. So this isn't some black box. There is a long history of peer-reviewed publications that got us to this point. The basic science behind these techniques has already been published. This is just the next step - presenting the clinical trial data for these studies.
They could put the information on their own webpage for all I care, as long as they are willing to stake their reputations on it.
The point is that we, the audience, do not have access to the requisite information (methods, data, background refs they trust) to even start attempting to judge the claims. Even if a conference talk/poster/abstract is available somewhere, that is an extremely limited format of communication.
Think of this as a preprint for the paper. It's just a talk.
If you want to blame anyone here, you could blame the media for reporting on a talk. But then again, the public likes to hear good news about cancer research, so I think the media is okay on this one.
I can assure you 100% this is not the case. Perhaps they could get that info by extensively talking to the presenter afterwards (which will not be recorded), but no way you can fit it into a 15-60 min talk or onto a poster, or any way that will be shared later. Not physically possible.
Edit: Ok, I imagined a bunch of androids like Data (star trek character), and believe it may be physically possible but would require superhuman capabilities to transmit and process information.
If you're not in that field, then no, you're not going to walk away with a good understanding of the work. Talks are always tailored to the audience.
Also, did the link change? I can't find the line I originally quoted anymore.