How we think about carrying evidence across country borders

The rapid emergence, and subsequent uptake of oncology real-world data (RWD) has ushered in a new age of cancer treatment — one where we are beginning to support real-time decision-making and empowering clinicians to be more efficient than ever before; however, in this new age we’ve found ourselves asking an age-old question: does the data we are using to help make these decisions truly reflect the patient in front of us? The question is more pressing when called to answer for the well-being of an entire population. Can the experiences of US patients tell a reliable story applicable to patients from other countries?

For health technology assessments (HTA), the question of data relevancy is at the forefront of decision-making. In these cases, decision makers are called to decide how well a given drug may perform in the real world, relative to the next best thing, and what a reasonable cost is for the added benefit for an entire population. So, ensuring that the population of interest is reflected in the data at hand is a crucial part of their process.

To illustrate the task, and it’s accompanying challenges, consider the fictional country of Transportugal, who has been presented with real-world evidence (RWE) from the US to support a label expansion for a product in their country. Though the US study was conducted with sufficient rigor, Transportugal’s subject matter expert is quick to point out some immediate differences between their country and the US, listed in Table 1, which cast doubts on the external validity of the study’s results.

Table 1: Differences in Cancer Care between the US and Transportugal*

*Transportugal is a fictional country used for exemplification

From the differences in Table 1, we can begin to understand why making decisions with evidence external from the population of interest is difficult. Don’t we expect differences in the site of administration to affect outcomes? And if the drug in question only accounts for 30% of the market share in the US, compared to the 50% in Transportugal, aren’t the cohorts fundamentally different?

In reality, the list of difficulties go on:

• Do the guidelines and treatment pathways differ from country-to-country?
• Are the same backbone therapies available to both groups of patients?
• Are the frequency of visits and diagnostic assessments comparable?

Questions, such as those provided above surrounding the relevance of US RWD to ex-US countries, are questions of uncertainty in the transportability (Figure 1) of inferences. Can the insights generated from the US transport to another country and allow for expedited decision-making and promote health equity both within the US and beyond?

Though there is debate in the literature, we can likely never say for sure whether a given inference is transportable. We can never know with certainty whether the group of patients for which we intend to make a decision will have the same outcomes in another setting, such as Transportugal,* as they do in the US even with the same technologies¹. A treatment provided to those with extensive caregiver support may realize more health gains than the same treatment provided to someone forced to provide for themselves; however, the data will view them equally. That is to say that in the data, those two patients are equal (eg, binary). In the data, the treatment was taken, or it was not, which makes transporting the true weighted effects difficult.

This is to say that with the results in-hand, one may find themselves pressed to retrofit them to a given target population. So then, our hope lies in the prespecification of our analysis with the goal of a representative population in mind. To help with this, we at Flatiron have created a list of considerations for the representativeness of US data for HTA decision-making (Table 2).

Using this framework, we can rethink how the study was pre-specified to prevent any undue anxieties thereafter. At a bare minimum, we can become more transparent about possible limitations to create an environment of trust, rather than the skepticism demonstrated by our fictional subject matter expert. For example, we probably could have identified differences in market share from the beginning of the analysis and knowing this may have changed how we constructed our cohort — to ensure those with access in transportugal are represented in the US EHR data. Or, we may have considered differences in site of administration and opted for a hierarchical model to handle these discrepancies.

Ultimately, there are a variety of ways that a target population may vary from that represented in any given dataset. The hope is that with the help of the considerations below, we can construct cohorts that are fit-for-purpose, identify populations similar to ex-US populations, and at the very least, be cognizant of the limitations that exist as we continue to push for a future where the full potential of RWD can be realized.

Table 2: Considerations for the Representativeness of US EHR Data for HTA Use Cases

	TRANSPORTABILITY ELEMENT	RATIONALE
	Patient Characteristic Differences
	Baseline demographics	Demographics may encompass a set of effect-modifying variables - differences in the prevalent and incident population should be considered
	Prevalence of disease	The baseline prevalence of a given disease may affect the transportability of some elements based on the mathematical association with relevant endpoints
	Preference for modifiable risk-factors	Preferences, and thus the prevalence, for modifiable risk factors (smoking, obesity, etc.) within a given population may modify the transportability of outcomes between countries if these risk-factors are known effect modifiers.
	Biomarker prevalence	For cancers with a diverse genetic etiology, there may exist significant treatment effect heterogeneity. Therapies indicated for those cancers may lack transportability in populations with a widely different biomarker makeup. Further, because biomarker testing rates may differ between populations, those selected into the cohort may also differ and affect the transportability of outcomes.
	Treatment Differences
	Access to a given treatment	A prevalent population may not be represented in the EHR data given restriction in access based on socio-economic or variability in payer preferences for a given product. Thus, patients selecting into a given cohort could vary and impact observed outcomes.
	Access to supportive care	Supportive care is known to improve outcomes for patients in many settings; however, access to supportive care varies within and between countries.
	Market share of the pharmaceutical(s) of interest and competitors	Environments with a large diversity of available technologies for a specific indication require contextualization for who selects into a cohort treated with a specific technology.
	Market share of backbone therapies used concomitantly with a therapy of interest	Even in situations where the market share for a technology of interest is the same, concomitant therapies of interest (eg. high versus low dose dexamethasone) may differ. If these therapies are effect-modifying, the distribution of them in the given data will affect transportability of the outcomes.
	Guideline differences between jurisdictions / localities	Because the approved label /reimbursement criteria for a given therapy may vary, the way a product is used between countries may also sometimes differ, which may present itself in what is known as the compound treatment problem. Further, labels may also influence the preceding drugs that patients have been exposed to, complicating the question of transportability.
	Setting Differences
	Treatment site variation	It is thought that outcomes between academic/research institutions may be different than those observed for community practices based on available resources. Thus, if countries differ in the distribution of these treatment sites for a given cohort of interest, outcomes would also be expected to differ.
	Differences in time-to-treatment initiation within a disease’s natural history	Time-to-treatment initiation may vary dramatically between countries (driven by localitied procedures to confirm diagnosis and/or healthcare system capacity) and therefore change a particular risk set and influence outcomes
	Disease assessment frequency	Disease assessment frequency can provide erroneous conclusions about metrics such as progression free survival or other outcomes that rely on monitoring schedules, and thus the time at which observations can be made.
	Preference for end of life care	In later lines of therapy, the risk-set a country chooses to treat may be different from that of another country based on differences in preferences for hospice. So, countries that tend to treat more aggressively may treat a sicker risk-set than that of a country that is more likely to choose for alternative end-of-life remedies

Notes: This table is intended to be a dynamic, living tool that will change over time. As Flatiron Health develops more learnings from experience with HTA use cases, and transportability nuances that arise, this tool will continue to be updated.