@jon_duke, thanks for leading this and great presentation. I think it’s a good opportunity for us to regularly revisit our community’s mission/vision/values to make sure they are aligned with the needs and interests of everyone involved. In general, I’m quite supportive of this new draft, but below are my specific comments to keep this thread alive.
Mission:
To improve health, by empowering a community to collaboratively generate the evidence that promotes healthier choices, and better care.
One thing about this mission is that it doesn’t explicitly state that we are trying to generate evidence from observational health data. My immediate reaction is that we do need to specify something along that lines in order to help rein in the scope just a little bit. In terms of intentions of our community, I don’t think we plan to conduct prospective randomized clinical trials to generate evidence, nor are planning to cut up mice or splice genes (I understand members in our community are doing translational research and other members of the community are interested in re-use of clinical trials, but I see both of those as potential external sources of hypotheses that could stimulate our database work but not our core function). In terms of words, I’ve been trying to use the term ‘observational health data’ to refer to non-interventional patient-level data captured in routine clinical care or from patients, which can include administrative claims, electronic health records, clinical registries, health/wellness surveys. In general, we’re focused on longitudinal patient-level data, but we have some data that is cross-sectional in nature, and that seems ok to me. We aren’t (yet) dealing with non-patient level data (either more granular, cellular stuff or more broad health-systems stuff). We also aren’t dealing with RCT data, though some have put RCT data into CDM. However we slice it, I think it’d be valuable for us to scope of what data we are dealing with to define the space we want to focus. Now, there are other terms that people sometimes use that I generally avoid: ‘electronic health data’ (if not electronic, what is it?, plus some people confuse this with only electronic health records), ‘clinical data’ (often confused with RCT data), ‘real-world data’ (chasing the fad, which for all I know will be over by the time I return from paternity leave), … I don’t have a strong vested interest in the word we choose to use, but i think defining the term and making it prominent throughout could have some utility to us.
If possible, I’d like to get some adjective in front of the word ‘evidence’, something like ‘reliable’ or ‘credible’. There’s lots of people doing observational analysis, but the vast majority is noise, not advancing health. we need to make the pursuit of ensuring reliability of the evidence central to our cause. i like the ‘reproducibility’ value, but i’d like to promote this idea into our mission somehow.
Vision:
We envision a world in which observational research produces a comprehensive understanding of health and disease.
The term ‘observational research’, I wonder if that’d be better to be something like ‘observational health data science’. Rationale: ‘research’ is bounded in some contexts (different from clinical operations) and some who know of our work from past OMOP data think of research as only methods research, without the development and application aspects of the OHDSI community’s focus.
‘understanding of health and disease’, I wonder if we may want to expand that a bit in a way that would help with narrowing focus a bit,. something like ‘understanding health, disease, and the use and effects of medical interventions’. I would like for us to clearly highlight that understanding treatments (when are they used? why are they used? which ones work?) is a focus above and beyond descriptive epidemiology of disease, I’m not restricting this to drugs, happy to talk about all health interventions that we observe in our data, but I think focusing on the points in care where medical decisions are made is likely more fruitful than just characterizing who has what ailment.
Explicitly stealing the idea that Jon threw out 2 years ago which I will continue to implore him to publish: doctors make treatment choices based on their education and past experience, and when there’s uncertainty, they ask colleagues about their experience and opinions. Observational data can be used to ‘crowdsource’ that ‘consultation’ process by allowing us to summarize what decisions all doctors make when faced with all different scenarios across all different types of patients. Without (yet) getting into causal inference, just understanding descriptively which patients are given which treatments at which moments in their health encounters based on which diagnoses and which baseline characteristics can have tremendous value. we should know who gets what treatments at what times for what purposes, and then we can start to ask the question of what happens to them afterwards.
Framed as a prediction problem, this can be: ‘what treatment choice is made at a moment in time, given all prior medical history?’ Framed as a descriptive problem, ‘for patients with a given treatment choice, what are the characteristics of those patients before/during/after the choice?’ With these in place, it’s easier to frame the estimation problem ‘what is the effect of treatment choice X on outcome Y?’ or the prediction problem of ‘what is the probability of outcome Y if I choose choice X?’
Values:
Innovation: Observational research is a field which will benefit greatly from disruptive thinking. We actively seek and encourage fresh methodological approaches in our work.
Love the value. I think it’d be useful to highlight that we see the need to improve the ‘science of observational analysis’ through innovation in the design of analytical methods and the management of observational data, but we’re also guided by new approaches to empirically evaluate the reliability of the methods, such that the research can be immediately translated into development that can have direct clinical applications that can be meaningfully interpretted.
Reproducibility: Accurate, reproducible, and well-calibrated evidence is necessary for health improvement.
Probably my top priority, reproducibility seems a high buzzword in the literature and grants right now, and i support it. but i wonder, is reproducibility simple one aspect of reliable evidence, or is it the other way around? The main objective is that we are generated evidence that all stakeholders can trust so they can confidently make better medical decisions. Knowing the study is reproducible increases its trustworthiness but I’m not sure if that alone is sufficient.
Community: Everyone is welcome to actively participate in OHDSI, whether you are a patient, a health professional, a researcher, or someone who simply believes in our cause.
Well stated. Maybe add ‘welcome and encouraged’
Collaboration: We work collectively to prioritize and address the real world needs of our community’s participants
‘prioritize and address’ sounds to me like the collaboration is only in identifying the problems. I think we want collaboration to be in all aspects of our work, including identifying the problem, research and developing an appropriate approach, and applying the solution to the problem across all appropriate data resources.
Openness: We strive to make all our community’s proceeds open and publicly accessible, including the methods, tools and the evidence that we generate.
‘proceeds’ made me think of money. perhaps another word, like ‘work products and contributions’ would be better?
Beneficence: We seek to protect the rights of individuals and organizations within our community at all times.
‘Beneficence’ wasn’t in my personal vocabulary, but now I like it:) Only comment here: does ‘right of individuals within our community’ include the patients whose data are in the observational health databases? It’d be nice if that could be more explicit somehow, we’re not just protecting our community, we’re protecting the patients whose data are being used to generate the evidence.
