This paper aims to discuss the consistency of conjoint estimation for a special approach called reversed conjoint approach, for economic questions in medical decision making. This approach is different from conventional conjoint design where only product attributes are compared in the study design, usually with paired comparisons. As the objective is to analyze the effects of economic questions on physicians’ preferences, with the use of this elicitation method of preferences, it deals with a value judgment on patients, including their socio- economic profiles.  It is based on a decompositional type of choice models, aiming to represent a comprehensive set of criteria and attributes to help the physician in his value judgment [1]. In such a case a full profile approach is more relevant to judge a patient’s health than using pairwise comparisons, therefore usually inconsistency indices may not apply [2,3].

As the approach used a Brunswik lens model, the first section of the paper provides some insights on how judgement research and the mathematical representation from psychological models (e.g. Tucker) can match the quantifications on the cognitive systems of judges with the ecology. However, clinical judgement studies usually do not represent patient or physician’ economics. The second section of the paper will show how usual axiomatic systems for conjoint models used for estimation of trade off choices between products or services on multi attributes and recently discussed by Brunelli [4] at the Eurogroup on MCDA may lead to useful consistency measures, for this specific conjoint approach. The paper will develop then an analysis on how special study designs of conjoint models, using cost cognitive cues, can help with incoherencies of impacts of budget restrictions implemented by public or private payers on populations; it also provides some venues for possible applications of operational research for management of health care budget and resources [1].

Clinical vignettes administered with pair of patient profiles on the same vignette were unsuccessfully to explore how patient economics influence physicians’ treatment choices in clinical practices.

Most conventional conjoint studies have provided metrics to measure physicians’ preferences between medical products and services, represented by attributes (safety, efficacy or price). However, for prices, the technique usually does not integrate cost to the patient or the health system or global copay level; so, using only a drug price or its level of copay is not enough.  In the conjoint model discussed in this paper, conversation on cost of care has been explored with qualitative research (mainly physician’s focus groups) to understand how and what economic information may influence physicians’ decision shifts in clinical practice. The criteria used in the conjoint study designs are then cost cues, representing economic concepts such as patient affordability or patient demand for cheaper clinical strategy. Therefore, the metrics uses unstructured data for critical decision points, where it is identified that conflicts or interference of economics may lead to aggravation of disease severity, additional costs for emergency visits, or other unnecessary costs.

This terminology of cues refers to the judgment research field (psychological models), which has its own representation of the ecology of a system (with tests of congruence). The system under study in this case, is the medical system; the sets of cues are clinical cues for medical tasks in clinical practices and cost cues when there are implicit restraints or interferences due to economics (for instance limiting patient adherence and compliance to guidelines). Cues either clinical or economic cues are then in the cognitive systems of the selected judges, the physicians, in this case.

Consistency issues of cost sensitivity indexes on physician’s choices

As the approach discussed in this paper comes originally from psychological models (mainly Lens models and their revisions (http://www.brunswisksociety.org), it is relevant to start with the way consistency is usually addressed with such decision models, based on judgment analysis. If the Lens model is used, the generation of attributes or patient cost cues comes mainly from unstructured data such as texts transcripts from conversation of care (mainly copayment) during the physician patient encounter; levels used in the conjoint models are identified from the decision shifts when physicians start thinking about cost and change their clinical strategies because of cost. Judgment research on cognitive systems using lens models relies on specific consistency test for cues which quantify the relationship with the ecology of the system under study (especially the congruence to have a study design as close as possible to the real context). In models of clinical decision making, the tests are used on physicians’ choices measured in experimental surveys or groups of healthcare professionals such as pharmacotherapy committees or drug review boards for listing decisions.

In the application discussed in this paper, tests address whether economic information measured with cost cognitive cues is consistent within and between samples of physicians in different national health systems. The context is the clinical setting in primary care with or without alternatives modes of delivery of services. The metrics used for the conjoint estimation is based on a conjoint survey to generate data for a cost sensitivity analysis on groups of physicians. Such cost studies can then be used for calibration of expenditures forecasting, as one type of discrete choice modeling, using various levels of analysis from individual to group and macro levels.

Judgment studies in medical sciences are useful to understand the differences of opinions either within a professional group (e.g. experienced physicians versus medical students) or between professionals (e.g. physicians, pharmacists, nurses). Such studies help with uncertainties in evidence in development and evolving medical policies (e.g. dealing with controversies in likelihood of treatment effects per subpopulations). For instance, the clinical controversy on rate versus rhythm in the case of atrial fibrillation (affirm trial) has been regularly debated with conflictual conclusions from evidence-based results on control of heat rate (correspondence approach) or from approaches aiming to restore good rhythm (“coherence approach”) [5,6]. The mathematical formulation used to test the reliability of such psychological Lens model in clinical judgment studies, as in other fields of sciences, is usually based on the Tucker formula and its revisions (Appendix 1). The tests for such clinical multi-cues systems have been developed especially since Hammond’ first contributions for the medical field [7]. The consistency test used in the Tucker formula is called Rs. It measures the consistency of judges’ responses to multi-cues for clinical tasks, between physicians in the relevant ecological system, the clinical system. However, cues representing cost or economics concepts are not under-researched in clinical judgment studies because of ethical and legal hurdles and there are limited designs with cost cues; experimental designs only include variations on clinical profiles in the selection of clinical vignettes, used for conjoint surveys.  In most cases, such conjoint valuation methods on physicians are used on product attributes for choices between different medical technologies [1].

Conjoint models belong to the categories of contingent valuation or stated preference studies; such methods are also used to investigate effects of physicians’ economics and not only product or patient economics: but mainly for impact of various compensation schemes or incentives (e.g. with DCEs). Interested readers can refer to recent meta-reviews on pay for performance.

The consistency issues discussed in this paper in conjoint models on physicians’ choices deal with both clinical and economic cues, called cost cognitive cues. For clinical cues, international consensus usually exists with clinical guidelines for different diseases exist; however, for economic cues, building a consensus is more complex since costs are strongly embedded in each medical system. It leads to first explore the influence of economics (both implicit and explicit information) with the cognitive systems of the decision makers in a medical system; it also requires integrating complex interactions and effects of resource allocation decisions (especially financial resources) inside the health system and possibly, outside the boundaries of traditional medical systems, in larger eco-systems with alternative modes of deliveries. However, similar economic concepts can also emerge from consensus building at international level such as affordability, ability to pay or demand for cheaper treatments. However, Rs consistency indices are usually lacking in judgement studies on economics and medical decision making. Therefore, it is useful to discuss whether other approach with axioms developed by mathematical economists on conventional conjoint models can be of use for this type of applications on health care.

Current axiomatic system for consistency of conjoint estimations [1]

Most axiomatic systems used for conjoint estimations address the preferences of a decision maker, for products or services compared with multi attributes. The research community in this type of Discrete Choice method has therefore developed series of inconsistency indices, mainly based on pairwise comparisons.

Interesting readers on a systematic review of consistency issue for conjoint studies in health care can refer to the review of 114 Discrete Choice Experimental studies in Health Care by De Bekker-Graub et al. [8] or to Ispor Task force report by Hauber et al. [9].

Such comparisons are mathematically represented by matrices providing the ratio of weights for each judgment on pairwise comparisons for a set of criteria. We mention in this paper an example of such an axiomatic system for consistency of pairwise comparisons [4] based on six properties on consistency and a seventh property on transitivity of value preferences. The six properties of the axiomatic system for ensuring consistency are the following:

P1: Test degree of inconsistency of preferences.

P2: Test stability, to test whether the consistency or inconsistency is stable.

P3: Test monotonicity and intensity

P4: Test monotonicity (case of quasi convex function)

P5: Test continuity

P6: Test invariance under inversion of preferences

P1 to P6 provides a good representation of a comprehensive set of axioms for consistency. It is used to quantify the inconsistencies of conjoint estimations. However, some limitations exist, especially concerning the following issue: when the first 5 properties for consistency indices are satisfied, then the transitivity property cannot be satisfied. Therefore, there is no existing function that can represent a conjoint estimation which is both consistent and transitive. Recent advances in this field however, lead to discuss some conditions that may allow both consistency and transitivity for a conjoint estimation [4]. A function capturing both consistency and transitivity depends on the system of axioms. If the property P5 on continuity is excluded of the list of axioms on consistency, then such function may exist. It will then depend whether the researcher believes the assumption of continuity can be excluded (in such a case, it means that small variations in preferences may not automatically lead to small variations in the consistency index). Such type of development on axiomatic systems may increase again the interest of conjoint estimations for assessing decision makers’ preferences [1].

In this paper, the special application of conjoint elicitation method of physicians ‘preferences and interferences with patient economics may benefit of the axiomatic approach just mentioned. The example provided may help to explore whether it is relevant to also develop such series of axioms for consistency of a representation of cost cognitive cues and its mathematical measurement. However, the original research mainly dealt with consistency of survey response for each physician, for original cost data from on physicians’ choices for different diseases. 

Consistency issues on physicians’ value judgment and cost cognitive cues

The original experimental survey that generated the cost cognitive cues emerged from an international group of researchers, through a consensus building process for six different national health systems (including their national system characteristics, especially cost sharing configurations). The economic topic investigated was the effect of patient charges on physicians ‘treatment choices in primary care settings. The objective of the consensus process was to reach common relevant concepts, representative of what economics interfere with physicians’ choices.

A theoretical discussion is out of the scope of this paper, however interested readers on discussion of axioms can refer for instance to early works of the 50s [10-12].

It appeared to be also relevant for North-American physicians (physicians’ focus groups on asthma, Boston, Huttin (Endepusresearch Inc., 2006) and a similar methodology was used by a Canadian team in Ontario to analyze formulary restrictions on drug budgets [13].

The conjoint design analyzed physician’s value judgment on a patient diagnosed with one condition, using variations of levels on a set of 4 cost cognitive cues, representing the cognitive system of physicians responding to patient economics and not only patient clinical profile. The four cues representing economic concepts in the context of each health system were the following:

Cue 1: Patient affordability.

Cue 2: Patient demand for cheaper treatment.

Cue 3: Patient co-medication for co morbidity.

Cue 4: Patient severity.

Source: Readers can refer to endep/biomed book for a description of all cues, Ios Press, 2003.

The efficiency of the conjoint design was tested with the D efficiency test, already discussed in a previous article [3]. However, the properties of the multi cost cues and the consistency between each individual physician was not discussed so far. Physicians were classified as cost sensitive, when statistically significant physicians ‘decision shifts were observed with the conjoint model run with the bundle of 4 cost cues (analysis of variance on clusters of cost sensitive and non-cost sensitive physicians).

However, the decision shifts occurred with some of the cue levels only, and with different magnitudes according to the selected sample of physicians in each health system (similar clinical vignettes were administered in each system). If the decision shift happens for a physician above a certain level for one cue, then he should be more sensitive, according to the cue, than another physician whose decision shift occurs for lower levels of the same cost cue. In this conjoint estimation, the physician intended to shift his decision because of cost cues, should be consistently more cost sensitive than the other physician.

The following discussion illustrates consistency issues on cost cues, by examining two physicians’ profiles for each cue separately with examples from systems were the cost studies were performed, and then consider pairs of cues where physicians, whatever the system, were the most sensitive: patient demand for cheaper treatment (Cue 2) and patient co medication for co morbidity (Cue 3).

Cue 2 representing patient demand was by far empirically statistically significant in the four systems under study. Two levels were chosen for the cue: 0 and 1. However, the value one was associated to a specific economic narrative in each system, depending on various types of signals detected by the physician during the encounter and reported in each national transcribed report. For instance, it could be a sentence such as “please can you prescribe a generic or cheaper medication”; so, the coding “1” represents any “expressed economic narrative demand” from the patient, during the conversation with his physician. All physicians whose decisions are influenced by the cue (level 1) are cost sensitive physicians, and it is straightforward to identify the consistency of their treatment choices with a cue 2, level 1.

Cue 3 representing copayment for co medications was also very statistically significative. It was coding with levels 0, 1, 2 and 3 and is more interesting to discuss in relation with consistency issues. For such a cue, the physician has some leeway to opt for clinical strategies using combination drug therapies that will limit the number of copays, knowing the comorbidity profile of each patient. The type of decision shifts then will depend on the type of combination therapies available, patient profiles, levels and ranges of copay already paid by the patient. We expect the physician will try to minimize the level of copay for his patient.

If the patient demand cheaper treatment (cue level 1), it is likely he will also tend to minimize the global copay, knowing the financial burden associated to the comorbidity of his patient.

However, according to the conditions, the physician may not automatically change the clinical strategy to lower the global copay. We can then examine what can be the possible reasons. It may be a clinical reason that he cannot switch to a different treatment; he may also be ignorant about the financials associated with the copayment associated with the comorbidity. Usually reminder systems on copays on drugs for instance, classify the drugs in different categories of reimbursement levels (e.g. proportion or percentage of drug prices, different tiers of copay such as specialty tiers versus different types of generics); it is unusual to compute such information, when treatment decisions happened.

If the clinical criterion is dominant in the reimbursement system design, then the physician will have more choice at individual level, to opt for a level of reimbursement associated with his clinical choice or a category especially within exemption categories, listed in the clinical priorities setting from the public or private payers. If the reimbursement criteria are more dominated by socio demographic criteria, the physician’s choice may be more constrained by the technology assessment agencies and the normative rules on cost effectiveness set up for the health system. 

Consistency issues discussed so far, refer to physicians’ preferences inside a health system. If this first level may lead to consistent behaviors of all physicians in the system, a second layer of consistency had also to be examined: namely how a national group of cost sensitive physicians’ responds to patient demand for cheaper treatment in comparison with a group from another system? Are the responses similar in term of treatment shifts (e.g. types of drugs in prescribing patterns) and is the magnitude (intensity) of responses very different? Then, the discussion moves to another level: an international comparison between health systems (or multi-health systems).

However, in the current valuation of health system framework, the rules of cost effectiveness are challenged by other stakeholders and the outcome research community responds by what is called the augmented cost effectiveness studies, which may also adjust threshold at different levels of aggregations.

Results tend to show that intention to treat (according to the responses of the conjoint models) may lead to opposite treatment shifts in term of pharmacotherapies in each system: more beta blockers in one health system, less beta blockers in a second health system; increase of cheaper diuretics and increase of beta blockers in one health system, very large increase of only cheaper diuretic in another health system. For this layer of analysis between systems, opposite treatment shifts were observed with the experimental studies administered in different health systems. Therefore the consistency issue in such type of conjoint estimation may need to be constrained within a health system first and the level of inconsistency observed between health systems (e.g. in the US or Canada, federal versus state/provincial jurisdictions for health insurance programmes) analyzed with the modeling of  health systems ‘characteristics and their effects on physicians ‘choices (including for instance, innovation adoption strategies in the health system, discounting practices and relative pricing for drug classes, variations in medical culture, different compensation schemes and financial incentives).

Cognitive architectures and cost modules on physicians’ choices

The development of decision tools or medical software that could integrate such sets of cost cognitive cues either on patients or physicians should also address the type of cognitive architecture relevant for medical informatics and decision ruling systems inside health care organizations such as teaching hospitals. Figure 1 illustrates the cognitive architecture that was considered in the original studies to integrate one or several modules of cost cues.

It was based on preliminary discussion with the Brunswik society in the USA. Early discussion with Professor Hammond, during the Brunswik meeting in Toronto,2005 led to consider for instance the ACT architecture developed in Pittsburgh by Anderson et al. [14] and Borst et al. [15], among relevant architectures from information processing theories. A review of recent development and use of such models of cognition is however beyond the scope of this paper. The application was also presented at that meeting, within a context of both a political and business intelligence, as a decision tool in interaction between economic stakeholders, mainly biopharmaceutical companies and policy makers. Physicians are critical in the supply and demand of such medical markets. The decision tool, using the algorithm for this application, would rely on similar types of information than marketing departments of companies; it may countervail their information with additional representations of the influence of patient economics on physicians’ treatment choices. In this case, the multi-cues approach, in addition to product economics, represents what type of economic information on patients ‘economics is processed by physicians, implicitly during a patient encounter. Currently, the development of frameworks for integrating more stakeholders may lead to consider, not only the clinical system, but larger systems and a role for integrators at different levels [16]. It also leads to consider system intelligence at state level [17] especially for investigations on budget restrictions in national socialized health care systems.

Among recent development of models of cognition based on the ACT-R cognitive architecture are the data driven model-brain mapping by Arrow et al. [11] or computational models using cognitive architecture ACT-R to implement effective e-learning systems [18].

LIMITATIONS AND FURTHER RESEARCH

The multi-cue approach used to analyze the impact of economics on physicians ‘choices appears very useful. So, research on best conditions to ensure consistency of studies is important. Other consistency issues should be addressed to develop an axiomatic system for sets of multi-cost cues useful for reimbursement designs. In such types of stated studies, monotonicity test is for instance often performed [19].

In the current stage of this type of research on cost cognitive cues and physicians’ choices, tests of monotonicity are missing. Only tests of stability of respondents have been performed in the series of cost sensitivity studies (stability of physicians’ responses with similar vignettes at different stages of the conjoint questionnaire were successful in all the systems, in the original endep/biomed cost studies, 2003). In further studies applied to cost cognitive cues in physicians ‘choice models, other types of comparisons of pairs of patient socio economic profiles using the set of four cognitive cues could be also tested with examples of dominant pair test comparisons).

This paper also aimed to discuss existing measures of consistency, either from judgment research or stated preference methods, in order to generate useful validated tools for decision ruling systems of medical groups. Further research will also need to discuss whether mathematical formulations used in Multi Criteria Decision Analysis (MCDA) can be relevant for such applications. For instance, the research conducted in the MCDA community, may contribute on best algorithms according to which criteria or groups of criteria are dominant among others. Already, the original research in Thailand, used as a model for WHO, selected multiplicative functions instead of additive functions, to aggregate criteria for committees’ decisions on essential drug lists for hospitals (Appendix 2). The development of mathematical formulations may also refer to the heuristic literature, especially on fuzzy sets [20,21] Current research also addresses the greater inconsistency associated with greater complexity of choice sets. For instance, sequential designs and heterogeneous semi-Bayesian designs are proposed [22].

CONCLUSION

Economic topics on physician-patient’s choices are usually mainly studied with revealed preference models, using effective data. However, problems of representativity of preferences for different types of providers and patient groups raise growing concern in the move towards precision medicine, fast digitalization and financing reforms. Therefore, adjustment methods using behavioral economics and additional algorithms, run with micro conjoint data or DCE data, to capture and measure more variations of risks and benefits per patient and the ways physicians ‘choices are influenced by economics are timely.

In this paper, multi cognitive cost cues systems are used to describe the way cost may interfere with medical decision making. This approach comes from psychological models of clinical judgment studies and conjoint statistical methods (among stated preference studies) and the paper shows how such multi cognitive cost cues systems may be useful in health financing systems.

Consistent measures from clinical judgment studies; for different physicians’ groups exist in the psychology literature (the Rs scores in medical sciences are usually very high in comparison with other fields such as education or business, according to recent meta-reviews by Karelaia and Hogarth [23] and Reips and Wittmann [24]. The multi-cues approach is also very useful to compare judgments of different groups of professionals, inside a health care organization: for instance, experienced MDs versus medical students, physicians versus nurses. In such cases, results can help to identify the differences in cognitive systems of different professions (most important cues in each profession [25]. But this approach may also need to be integrated within more comprehensive value assessment frameworks, with multi stakeholders inside and outside the medical system. The different methodologies, especially the ones used in MCDA, may therefore be relevant [26]. Moreover, economic or cost cues are usually not measured in clinical judgment studies and additional designs still need to address the ethical and legal conflicts.  

The integration of various stakeholders ‘perspectives, at national and international levels with adjustments of health budget and welfare contracts, may therefore lead to compare additional axiomatic systems, not limited to the consistency measures on cues, measured for one or two groups of judges, with mathematical formulae’s such as the Tucker formulae. More research is needed, especially for further integration of DCE studies in economic models and decision-making frameworks [27,28].