The Effects Table (ET) has become one of the most common tools in the area of Benefit-Risk Assessment. It displays a concise summary of the key benefits and risks of a new product compared to either placebo or an existing product. See here on more insights into what an effects table is and why it is helpful.
On initial inspection, it may look simple to create. However, those who have already tried to do the exercise know that it is not at all easy.
So, What challenges would I come across when creating an Effects Table?
In 2014, we – the EFSPI Benefit-Risk Task Force – performed a survey among pharmaceutical companies, academic institutions and Contract Research Organizations (CROs) to provide feedback on their experience with creating an ET.
From the 16 invited organisations (12 pharma, 3 academic, 1 CRO), 8 responded to provide feedback (7 pharma and 1 academic) and there was a total of 10 respondents (given three members of one organization provided feedback).
All respondents had a good understanding of structured BRA (SBRA), expressing prior experience in constructing an ET or, if they have not constructed one formally, they had experienced much of the thinking to pull together benefits and risks of a medicine.
We asked the surveyees for hurdles they had faced for constructing an ET and the most common theme expressed in the free text field was around the exercise being time consuming to perform, especially in the context of selecting outcomes to include in the table. Among the 10 respondents, 9 (90%) mentioned “time to agree” as a hurdle and 8 (80%) expressed difficulties in endpoint selection.
So, Which endpoints should we include?
Our advice: Keep it simple by including the minimum number of endpoints in the ET needed to effectively convey the message. Only include effects (whether considered “known” or “potential”) that could influence the benefit-risk balance or that are judged to be important even if no apparent treatment difference is observed; i.e. this could include risks such as QT prolongation, or outcomes of specific interest to the therapeutic area such as suicidal risk with antidepressants, etc.
And, How do I select the right endpoints?
At a general level, be ready to have extensive discussions but it is a necessary step to come to an agreement.
We listed some points to consider below to help you:
Who should be involved in the discussion?
Our advice: You need to have a multi-disciplinary team with people representing different perspectives on the drug. It brings transparency and objectivity to the discussion. This team should include at least representations from the company’s medical, safety, regulatory and statistics functions.
However, facilitation of a cross-functional team can be a challenge. Having either an internal BR scientist or a small panel of internal BR experts can facilitate the process, an approach that the survey highlighted as being used successfully in some organisations.
Should an external perspective be sought?
Our advice: Independent expert input can be very helpful for the sponsor as it serves to bring in an external perspective. This can be particularly useful when the drug(s) under considerations don’t belong to an established class. Furthermore, patients’ and prescribers’ views are increasingly being used to add value for the SBRA. Interview sessions with patients and treating physicians and/or surveys can be conducted to support the selection and priority ranking of endpoints.
When is the right time to take action?
Our advice: To avoid potential bias, we encourage teams to agree on the endpoints and structure of the ET sooner rather than later. We recognize that this is only to a certain extent possible as knowledge of an experimental medicine’s benefits and risks evolves during the development program, especially with respect to safety. However, it allows early alignment on the development frame (target indication, patient population, medical needs, treatment landscape etc.) and anticipated key benefits and risks ; it allows to proactively address the anticipated gaps and avoid protracted discussions at a later stage when the team is under significant time pressures and constraints. Thus, starting the process at an early stage of development (e.g. after a commit to medicines development milestone has been achieved) and refining it through the drug’s lifecycle will bring both objectivity and efficiency.
And, in practice, how can I reduce the number of endpoints?
Our advice: Building and pruning a value tree is a good approach to facilitate selection. Two survey respondents reported creating a value tree and then “pruning” it to get to an agreement on the core benefits and risks.
Moreover, one should consider excluding overlapping endpoints. Including endpoints that carry overlapping information (e.g. serious adverse event and death since any death is, by definition, a serious adverse event) can indeed lead to double counting. However, this practice is controversial. While the inclusion of endpoints that are overlapping or preference-dependent is worrisome for quantitative models such as MCDA or QALYs that assume preference-independency, it may be acceptable for an ET that is being used within a purely qualitative framework since it is ultimately the readers’ judgment call to conclude whether the visualized attributes provide additional or complementary information.
On another matter, it would be erroneous to exclude important preference-substitute endpoints from the ET purely because they are statistically correlated to each other. For example if a drug is able to have different effects such as reduction of cholesterol and reduction of blood pressure, the fact that this happens with high correlation (i.e. the subjects with the greater cholesterol reductions are the subjects with greater blood pressure reductions) is not a sound reason to exclude one of these two endpoints from the ET. A message can be provided where overlapping or correlation could occur in the ET.
The EFSPI Benefit-Risk Task Force would like to thank those who responded to the survey and contributed by sharing their valuable insights into the challenges of constructing an ET.
This article was written by Veronique Robert, Delyth Jones, Alberto Garcia Hernandez, and Alexander Schacht