Skip to main content
Background: Scientific hypothesis generation is a critical step in scientific research that determines the direction and impact of any investigation. Despite its vital role, we have limited knowledge of the process itself, thus hindering... more
Background: Scientific hypothesis generation is a critical step in scientific research that determines the direction and impact of any investigation. Despite its vital role, we have limited knowledge of the process itself, thus hindering our ability to address some critical questions. Objective: This study aims to answer the following questions: To what extent can secondary data analytics tools facilitate the generation of scientific hypotheses during clinical research? Are the processes similar in developing clinical diagnoses during clinical practice and developing scientific hypotheses for clinical research projects? Furthermore, this study explores the process of scientific hypothesis generation in the context of clinical research. It was designed to compare the role of VIADS, a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies, and the experience levels of study participants during the scientific hypothesis generation process. Methods: This manuscript introduces a study design. Experienced and inexperienced clinical researchers are being recruited since July 2021 to take part in this 2×2 factorial study, in which all participants use the same data sets during scientific hypothesis-generation sessions and follow predetermined scripts. The clinical researchers are separated into experienced or inexperienced groups based on predetermined criteria and are then randomly assigned into groups that use and do not use VIADS via block randomization. The study sessions, screen activities, and audio recordings of participants are captured. Participants use the think-aloud protocol during the study sessions. After each study session, every participant is given a follow-up survey, with participants using VIADS completing an additional modified System Usability Scale survey. A panel of clinical research experts will assess the scientific hypotheses generated by participants based on predeveloped metrics. All data will be anonymized, transcribed, aggregated, and analyzed. Results: Data collection for this study began in July 2021. Recruitment uses a brief online survey. The preliminary results showed that study participants can generate a few to over a dozen scientific hypotheses during a 2-hour study session, regardless of whether they use VIADS or other analytics tools. A metric to more accurately, comprehensively, and consistently assess scientific hypotheses within a clinical research context has been developed.
The book reports on the current state on HCI in biomedicine and health care, focusing on the role of human factors, patient safety well as methodological underpinnings of HCI theories and its application for biomedical informatics.... more
The book reports on the current state on HCI in biomedicine and health care, focusing on the role of human factors, patient safety well as methodological underpinnings of HCI theories and its application for biomedical informatics. Theories, models and frameworks for human-computer interaction (HCI) have been recognized as key contributors for the design, development and use of computer-based systems. In the clinical domain, key themes that litter the research landscape of health information technology (HIT) are usability, decision support and clinical workflow all of which are affected directly or indirectly by the nature of HCI. While the implications of HCI principles for the design of HIT are acknowledged, the adoption of the tools and techniques among clinicians, informatics researchers and developers of HIT are limited. There is a general consensus that HIT has not realized its potential as a tool to facilitate clinical decision-making, the coordination of care and improves patient safety. Embracing sound principles of iterative design can yield significant dividends. It can also enhance practitioners abilities to meet meaningful use requirements. The purpose of the book is two-fold: to address key gaps on the applicability of theories, models and evaluation frameworks of HCI and human factors for research in biomedical informatics. It highlights the state of the art, drawing from the current research in HCI. Second, it also serves as a graduate level textbook highlighting key topics in HCI relevant for biomedical informatics, computer science and social science students working in the healthcare domain. For instructional purposes, the book provides additional information and a set of questions for interactive class discussion for each section. The purpose of these questions is to encourage students to apply the learned concepts to real world healthcare problems.
The study of medical expertise has been the subject of research since about the mid-1970s. The approach has drawn extensively on cognitive methods of protocol analysis, but has also appropriated methods and theories from the study of... more
The study of medical expertise has been the subject of research since about the mid-1970s. The approach has drawn extensively on cognitive methods of protocol analysis, but has also appropriated methods and theories from the study of memory, language comprehension, visual perception, and human–computer interaction. Medicine is a knowledge-rich and ill-structured domain that provides a rich vehicle for studies of complex cognition. It is a highly stratified discipline and we can chart progressions of expertise from the novice medical student to the seasoned board certified medical specialist. It is also a well-differentiated field with substantial specialization and distinct bodies of knowledge. There are two broad categories of research in the study of medical cognition, research targeted at characterizing the structure and use of basic science knowledge and research investigating the process of clinical reasoning. The studies range from experimental tasks representative of medical practice to studies of real-time decision making in high velocity medical environments. This article is concerned with the characterization of medical expertise and its development. Specifically, the chapter discusses research, theories and methods pertaining to the study of comprehension of biomedical concepts, clinical reasoning, and decision making.
Critical Care Medicine www.ccmjournal.org 163 12. Chicotka S, Rosenzweig EB, Brodie D, et al: The “central sport model”: Extracorporeal membrane oxygenation using the innominate artery for smaller patients as bridge to lung... more
Critical Care Medicine www.ccmjournal.org 163 12. Chicotka S, Rosenzweig EB, Brodie D, et al: The “central sport model”: Extracorporeal membrane oxygenation using the innominate artery for smaller patients as bridge to lung transplantation. ASAIO J 2017; 63:e39–e44 13. Ranney DN, Benrashid E, Meza JM, et al: Central cannulation as a viable alternative to peripheral cannulation in extracorporeal membrane oxygenation. Semin Thorac Cardiovasc Surg 2017; 29:188–195 14. Hodgson CL, Stiller K, Needham DM, et al: Expert consensus and recommendations on safety criteria for active mobilization of mechanically ventilated critically ill adults. Crit Care 2014; 18:658 15. Turner DA, Cheifetz IM, Rehder KJ, et al: Active rehabilitation and physical therapy during extracorporeal membrane oxygenation while awaiting lung transplantation: A practical approach. Crit Care Med 2011; 39:2593–2598
The past few decades have produced a cumulative body of experiential and practical knowledge about user experience, adoption and implementation to guide future design work. However, knowledge solely based on practical experience or... more
The past few decades have produced a cumulative body of experiential and practical knowledge about user experience, adoption and implementation to guide future design work. However, knowledge solely based on practical experience or empirical studies are not adequate to account for the immense variety of health information technologies and the rich array of contexts that constitute the practice of medicine. Theoretical knowledge is needed to develop a more complete understanding of the user experience as well as to produce robust generalizations and sound design principles. In this chapter, we describe theories of cognition in the context of human-computer interaction (HCI) in healthcare domains. In the first sections, we review the family of theories subsumed by information-processing approach. The subsequent sections focus on theories of external cognition and distributed cognition, which greatly expand the focus of HCI framework for healthcare contexts. This chapter also examines related research investigations and draws implications for the science of HCI and foundations of design. Although cognitive theory cannot provide all of the needed insights, it remains a powerful tool for advancing knowledge and furthering the pursuit of science.
Sexual decision-making in young adults continues to present challenges for public health and social science researchers more than three decades after the first HIV epidemic. The objective of this chapter is to identify relevant social and... more
Sexual decision-making in young adults continues to present challenges for public health and social science researchers more than three decades after the first HIV epidemic. The objective of this chapter is to identify relevant social and cognitive factors that may influence decision-making about risky sexual behavior in young adults. More specifically, the chapter focuses on our research on sexual decision-making among heterosexual urban college students in a high-risk HIV/AIDS community. Participants completed two weeks of daily sexual encounter diaries, followed by in-depth interviews. Both quantitative and qualitative methods were used to analyze the data. Results showed that participants’ decisions about the use of condoms varied by particular social factors, such as partner influence and condom use attitudes; and cognitive factors, such as perceptions of relationship status and HIV/STD risk assessment. Participants’ justification of their behavior differentiated higher- from l...
Objective IBM(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options... more
Objective IBM(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice. Methods This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH’s institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage. Results Nearly 60% (187) of 313 treatment p...
e14114 Background: Watson for Oncology (WfO) is an artificial intelligence-based clinical decision-support system which provides therapeutic options and associated scientific evidence to cancer-treating physicians. Oncologists at... more
e14114 Background: Watson for Oncology (WfO) is an artificial intelligence-based clinical decision-support system which provides therapeutic options and associated scientific evidence to cancer-treating physicians. Oncologists at Bumrungrad International Hospital (BIH) have used WfO since 2015. We examined the association between concordance of WfO therapeutic options and BIH treatment decisions with short-term clinical outcomes for lung cancer patients. Methods: This study included lung cancer patients seen at BIH for treatment and follow-up care and for whom WfO was used from 2015 to 2018. Charts were reviewed for concordance with WfO, documentation of disease progression, response to treatment, and survival. We evaluated concordance between oncologists’ treatments and therapeutic options listed as “recommended” by WfO. We evaluated association between WfO concordance and partial or complete response rates over a 24-month period by comparison of proportions with odds ratio. Progre...
95 Background: Watson for Oncology (WFO) is an artificial intelligence (AI) based clinical decision-support tool trained by Memorial Sloan Kettering. This retrospective observational study of breast, lung, colon and rectal cancer examined... more
95 Background: Watson for Oncology (WFO) is an artificial intelligence (AI) based clinical decision-support tool trained by Memorial Sloan Kettering. This retrospective observational study of breast, lung, colon and rectal cancer examined the concordance of treatment options provided by WFO to treatments selected by clinicians at Bumrungrad International Hospital (BIH) as a function of stage or cancer type. Methods: Concordance between WFO treatment options and treatments selected by BIH clinicians (WFO-BIH concordance) was defined as identical or equally acceptable treatments, as determined by a panel of experts blinded to the source of treatment. Relationships between stage or type of cancer and WFO-BIH concordant treatments were evaluated by Chi-squared analysis. Results: Analysis revealed a statistically significant association ( P = 0.02) between cancer stage and concordance. For all 4 cancer types combined, stages I-III demonstrated higher concordance than stage IV. A highly s...
Background Comparison of options from clinical decision-support (CDS) systems and decisions made in practice may be biased towards the treating institution. In this retrospective study, bias was minimized by blinding evaluators to the... more
Background Comparison of options from clinical decision-support (CDS) systems and decisions made in practice may be biased towards the treating institution. In this retrospective study, bias was minimized by blinding evaluators to the source of treatment recommendations, either Watson for Oncology® (WFO®) or treatments patients received at Bumrungrad International Hospital (BIH), a user of WFO®. Methods Treatments given were compared to therapeutic options provided by WFO®. Treatments that were identical to WFO® “recommended” (green, acceptable) were not evaluated further. Paired treatments were evaluated independently in a blinded fashion by each oncologist before consensus ranking of each pair as either acceptable, acceptable alternatives, or unacceptable treatment. The consensus for each treatment was compared to WFO®, with WFO® “for consideration” (yellow, acceptable alternative), and “not recommended” (red, unacceptable). Chi-squared tests analyzed the association between risk ...
6553 Background: Clinical decision-support systems (CDSS) such as Watson for Oncology (WFO) may reduce treatment variation in oncology, provided options offered by the system are at least as acceptable as expert, evidence-based options.... more
6553 Background: Clinical decision-support systems (CDSS) such as Watson for Oncology (WFO) may reduce treatment variation in oncology, provided options offered by the system are at least as acceptable as expert, evidence-based options. Deviation from expert consensus in practice is not well documented. In this blinded study, WFO therapeutic options and treatment decisions made by individual oncologists at Bumrungrad International Hospital (BIH) were evaluated by expert panel. Methods: Treatments selected by BIH that were labeled as either “for consideration” or “not recommended” by WFO were evaluated by a panel of 3 oncologists in 2018. The panel evaluated WFO options and previous BIH treatments for prospective cases from 2016-2018, blinded to the source of treatment option. Consensus of panel rated treatment pairs as: identical; both acceptable and roughly equivalent; both acceptable, but one preferred; one is acceptable and the other, unacceptable; neither is acceptable. The resu...
Background: Clinical decision support aids such as computerized weaning protocols (CWPs) aim to reduce medical errors and improve patient safety. However, the dynamic nature of critical care environments demands context-specific and... more
Background: Clinical decision support aids such as computerized weaning protocols (CWPs) aim to reduce medical errors and improve patient safety. However, the dynamic nature of critical care environments demands context-specific and complexity -inclusive assessment of these support tools for optimal results.
The study of diagnostic expertise initially focused on characterizing the reasoning process and, later, on understanding the nature of expert knowledge and its impact on performance, including memory, comprehension, and reasoning. This... more
The study of diagnostic expertise initially focused on characterizing the reasoning process and, later, on understanding the nature of expert knowledge and its impact on performance, including memory, comprehension, and reasoning. This research demonstrated that medical expertise is not a simple construct, and its development is characterized by non-linear growth in skills and knowledge. Facilitated by new technology, recent research has moved toward real-world studies (or a combination of both laboratory-based and naturalistic studies), with automated and often precise methods of data collection and analysis. Findings from these studies contribute to our knowledge of medical experts’ performance, which often varies as a function of domain complexity as well as the clinician’s ability to recognize and correct medical errors. The role of cognitive science in modeling diagnostic reasoning plays a significant role when building successful systems to support reasoning processes of indiv...
Objective With federal mandates and incentives since the turn of this decade, electronic health records (EHR) have been widely adopted and used for clinical care. Over the last several years, we have seen both positive and negative... more
Objective With federal mandates and incentives since the turn of this decade, electronic health records (EHR) have been widely adopted and used for clinical care. Over the last several years, we have seen both positive and negative perspectives on its use. Using an analysis of log files of EHR use, we investigated the nature of EHR use and their effect on an emergency department's (ED) throughput and efficiency. Methods EHR logs of time spent by attending physicians on EHR-based activities over a 6-week period (n = 2,304 patients) were collected. For each patient encounter, physician activities in the EHR were categorized into four activities: documentation, review, orders, and navigation. Four ED-based performance metrics were also captured: door-to-provider time, door-to-doctor time, door-to-disposition time, and length of stay (LOS). Association between the four EHR-based activities and corresponding ED performance metrics were evaluated. Results We found positive correlation...
Objective Over the last decade, electronic health records (EHRs) have shaped clinical practice. In this article, we investigated the perceived effects of EHR use on clinical workflow and meaningful use (MU) performance metrics. Materials... more
Objective Over the last decade, electronic health records (EHRs) have shaped clinical practice. In this article, we investigated the perceived effects of EHR use on clinical workflow and meaningful use (MU) performance metrics. Materials and Methods Semistructured interviews were conducted with 20 (n = 20) physicians at two urban emergency departments. Interview questions focused on time spent on EHR use, changes in clinical practices with EHR use, and the effect of MU performance metrics on clinical workflow. Qualitative coding using grounded theory and descriptive analyses were performed to provide descriptive insights. Results Physicians reported that EHRs improved their clinical workflow, especially on MU-related activities including door-to-doctor time and admit decision time. EHR use also affected physicians work efficiency, quality of care provided, and overall patient safety. Conclusion Physicians' perception of EHRs is likely to influence their practices. With negative ...
Despite the increasing global interest in information technology among health care institutions, little has been discussed about its importance for the effectiveness of knowledge management. In this study, economic theories are used to... more
Despite the increasing global interest in information technology among health care institutions, little has been discussed about its importance for the effectiveness of knowledge management. In this study, economic theories are used to analyze and describe a theoretical framework for the use of information technology in the exchange of knowledge. The analyses show that health care institutions would benefit from developing global problem-solving collaboration, which allows practitioners to exchange knowledge unrestricted by time and geographical barriers. The use of information technology for vertical integration of health-care institutions would reduce knowledge transaction costs, i.e. decrease costs for negotiating and creating communication channels, and facilitating the determination of what, when, and how to produce knowledge. A global network would allow organizations to increase existing knowledge, and thus total productivity, while also supporting an environment where the ge...
Numerous research and development projects have been aimed at implementing computerized patient record (CPR) systems. Yet little emphasis has been placed on physicians' ability to learn and use these systems or on their effects on... more
Numerous research and development projects have been aimed at implementing computerized patient record (CPR) systems. Yet little emphasis has been placed on physicians' ability to learn and use these systems or on their effects on physicians' reasoning. This article describes ...
SummaryBackground: Biomedical informatics is a broad discipline that borrows many methods and techniques from other disciplines.Objective: To reflect a) on the character of biomedical informatics and to determine whether it is... more
SummaryBackground: Biomedical informatics is a broad discipline that borrows many methods and techniques from other disciplines.Objective: To reflect a) on the character of biomedical informatics and to determine whether it is multi-disciplinary or inter-disciplinary; b) on the question whether biomedical informatics is more than the sum of its supporting disciplines and c) on the position of biomedical informatics with respect to related disciplines.Method: Inviting an international group of experts in biomedical informatics and related disciplines on the occasion of the 50th anniversary of Methods of Information in Medicine to present their viewpoints.Results and Conclusions: This paper contains the reflections of a number of the invited experts on the character of biomedical informatics. Most of the authors agree that biomedical informatics is an interdisciplinary field of study where researchers with different scientific backgrounds alone or in combination carry out research. Bi...
BACKGROUND We are studying ways to provide automated, context-specific links (called "infobuttons") between clinical information systems (CIS) and other information resources available on the World Wide Web. As part of this... more
BACKGROUND We are studying ways to provide automated, context-specific links (called "infobuttons") between clinical information systems (CIS) and other information resources available on the World Wide Web. As part of this work, we observed the information needs that arose when clinicians used a CIS and we classified those needs into generic questions. We then sought general methods for accessing information resources to answer the questions. METHODS For each generic question, we identified a satisfactory resource and then developed a method for retrieving from it the information relevant to the question. We then studied these methods to characterize them into general approaches. RESULTS We identified six general approaches and describe them in detail. These approaches range in complexity from simple, hard-coded links to intelligent agents and calculators. CONCLUSION Web-based information resources can be exploited using a relatively small number of methods, although the ...
Clinical guidelines aim to eliminate clinician errors, reduce practice variation, and promote best medical practices. Computer-interpretable guidelines (CIGs) can deliver patient-specific advice during clinical encounters, which makes... more
Clinical guidelines aim to eliminate clinician errors, reduce practice variation, and promote best medical practices. Computer-interpretable guidelines (CIGs) can deliver patient-specific advice during clinical encounters, which makes them more likely to affect clinician behavior than narrative guidelines. To reduce the number of errors that are introduced while developing narrative guidelines and CIGs, we studied the process used by the ACP-ASIM to develop clinical algorithms from narrative guidelines. We analyzed how changes progressed between subsequent versions of an algorithm and between a narrative guideline and its derived clinical algorithm. We recommend procedures that could limit the number of errors produced when generating clinical algorithms. In addition, we developed a tool for authoring CIGs in GLIF3 format and validating their syntax, data type matches, cardinality constraints, and structural integrity constraints. We used this tool to author guidelines and to check ...
This paper describes an approach to the evaluation of health care information technologies based on usability engineering and a methodological framework from the study of medical cognition. The approach involves collection of a rich set... more
This paper describes an approach to the evaluation of health care information technologies based on usability engineering and a methodological framework from the study of medical cognition. The approach involves collection of a rich set of data including video recording of health care workers as they interact with systems, such as computerized patient records and decision support tools. The methodology can be applied in the laboratory setting, typically involving subjects "thinking aloud" as they interact with a system. A similar approach to data collection and analysis can also be extended to study of computer systems in the "live" environment of hospital clinics. Our approach is also influenced from work in the area of cognitive task analysis, which aims to characterize the decision making and reasoning of subjects of varied levels of expertise as they interact with information technology in carrying out representative tasks. The stages involved in conducting c...
ABSTRACT Enormous advances in information technology have permeated essentially all facets of life in the past two decades. Formidable challenges remain in fostering tools that enhance productivity but are sensitive to work practices.... more
ABSTRACT Enormous advances in information technology have permeated essentially all facets of life in the past two decades. Formidable challenges remain in fostering tools that enhance productivity but are sensitive to work practices. Cognitive Informatics (CI) is the multidisciplinary study of cognition, information and computational sciences that investigates all facets of human computing including design and computer-mediated intelligent action, thus is strongly grounded in methods and theories from cognitive science. As an applied discipline, it has a close affiliation with human factors and human-computer interaction, and provides a framework for the analysis and modeling of complex human performance in technology-mediated settings and contributes to the design and development of better information systems. In recent years, CI has emerged as a distinct area with special relevance to biomedicine and health care. In addition, it has become a foundation for education and training of health informaticians, the Office of the National Coordinator for Health Information Technology initiating a program including CI as one of its critical elements to support health IT curriculum development. This book represents a first book on cognitive informatics and focuses on key examples drawn from the application of methods and theories from CI to challenges pertaining to the practice of critical-care medicine (CCM), including emergency medicine (ER), given the nature of complexity. Technology is transforming critical care workflows and re-organizing patient care management processes. CCM and ER have proven to be fertile test beds for theories and methods of cognitive informatics. CI, in turn, has contributed much to our understanding of the factors that result in complexity and patient errors.
Critical care environments are information-intensive environments where effective decisions are predicated on successfully finding and using the 'right information at the right time'. We characterize the differences in processes... more
Critical care environments are information-intensive environments where effective decisions are predicated on successfully finding and using the 'right information at the right time'. We characterize the differences in processes and strategies of information seeking between residents, nurse practitioners (NPs), and physician assistants (PAs). We conducted an exploratory study in the cardiothoracic intensive care units of two large academic hospitals within the same healthcare system. Clinicians (residents (n=5), NPs (n=5), and PAs (n=5)) were shadowed as they gathered information on patients in preparation for clinical rounds. Information seeking activities on 96 patients were collected over a period of 3 months (NRes=37, NNP=24, NPA=35 patients). The sources of information and time spent gathering the information at each source were recorded. Exploratory data analysis using probabilistic sequential approaches was used to analyze the data. Residents predominantly used a pati...

And 316 more