i do not have permissions to give you the exact number, but I can tell you I see >10,000 people. And see you get >4,000 cases. It is likely that the 3 where you see a 0% are US-based or elsewhere-based claims data sources.
The motivation behind my question is that in some healthcare systems people are asked to report all their health problems, etc upon registration. This is common practice in UK, Spanish, and Italian primary care, and it is to be expected that some healthcare professionals will use the “right” or “best” code, which in this case could be the “H/O” or related
Although this might be less relevant for other types of studies, I am not ready to dismiss this information for descriptive epi or drug utilisation studies, where e.g. a patient prescribed a med for migraine prophylaxis on registration (e.g. because they transferred out from somewhere else) would appear not to have an indication for that med if we dismiss the ‘H/O’ code
I think you answered your own question. As the researcher who decided that sensitivity errors is a bigger problem than index date misclassification errors. So you may want to use a cohort definition that has the h/o code in the entry event definition.
The problem with history of migraine code is that - all it means is we do not know the the onset date i.e. cohort_start_date. You are getting a date sometime after the disease started. It becomes your study limitation.
Mm… I disagree. My whole motivation for “phenotype flavours” at the top of this thread is that there are use cases where the inclusion of certain codes (here the “h/o” toy example) does in fact reduce error
E.g. In a prevalence study, as your characterization demonstrates, the exclusion of H/O migraine underestimates prevalence. I do not see how this is an error, because this use case does not care about the date when you were diagnosed with migraine
And this is just an example. There are others like drug utilisation studies where you want to assess indications, or the longitudinal modelling of comorbidity or frailty
@Daniel_Prieto The motivation behind my question is that in some healthcare systems people are asked to report all their health problems, etc upon registration
This was addressed in a 2005 paper by Jim Lewis at Penn. Differences id health care delivery systems may/will affect how information is recorded. If you develop a comorbidity score that only looks back 1yr and captures diagnoses and apply that to a US claims you get good capture since every time you come in we bill for diabetes. If you apply this same restrictive one year look back to an EHR system that might have a one code chronic diagnoses code recorded on a problem list you would miss this comorbidity. Underlying data and underlying questions do mater in application of phenotypes.
So - i am thinking in terms of how OHDSI defines an instantiated cohort table. It would look something like this
cohort_definition_id
subject_id
cohort_start_date
cohort_end_date
1
1
1/1/2015
9/15/2016
1
2
2/15/2015
8/15/2015
1
3
2/28/2015
10/15/2016
1
4
3/15/2015
9/16/2015
Now we have to use this for calculating prevalence or incidence. If we are calculating incidence we take cohort_start_date, if we are doing prevalence we can take any date.
Prevalence may be reported as a percentage (5%, or 5 people out of 100), or as the number of cases per 10,000 or 100,000 people. The way prevalence is reported depends on how common the characteristic is in the population.
There are several ways to measure and report prevalence depending on the timeframe of the estimate.
Point prevalence is the proportion of a population that has the characteristic at a specific point in time.
Period prevalence is the proportion of a population that has the characteristic at any point during a given time period of interest. “Past 12 months” is a commonly used period.
Lifetime prevalence is the proportion of a population who, at some point in life has ever had the characteristic.
We can calculate the period prevalence and the period is the 12 months of calendar year 2015 - all of the persons above would be eligible for numerator. If we wanted 2014 - none of them would be eligible.
If we have index date misclassification - there would be no problem for 2015 calendar year. We will have a problem for 2014 calendar year.
If a person enters the cohort because they had ‘history of migraine on 1/1/2015’ their cohort_start_date is still 1/1/2015. So they are not contributing to 2014 computation - while semantically we dont know when they truly started to have migraine, because all we know on 1/1/2015 is that they have history of migraine. So we have sensitivity error for 2014.
That yet again reminded me…do you guys still distinguish between target/outcome/feature cohort? for example, if the cohort of migraine is a feature cohort that will be used for, say, Table 1, wouldn’t we include h/o of migraine since we are tolerant to index event misspecification and care more about sensitivity?
A side note: if you’re still working on that big phenotype paper, it would be nice to include all of the above in it (nice figures, yesss!). And if any help is needed, I’d love to contribute
The way I think about it in your specific example is that I would exclude the patients with prior outcome on the cohort or analysis level. And then E11.2 would go into the outcome list and everybody with those codes prior to the index date would be excluded. Now, if that’s the only diabetes-related code a person has, I’d explore it more as this patient is suspicious to begin with.
In other words, potential causal relationship should be handled either at the analysis stage or manually (eg full-blown chart adjudication). That’s my take which can be different from the line of the party
Given a target (clinical description of migraine) we develop one or more cohort definitions (like case definition but at the population level). Each of this cohort definition is a model with different errors - that we currently describe mostly in sensitivity, specificity, date misclassification. The cohort definition tries to give us a persons best cohort_start_date to cohort_end_date - if it is off, its an error.
If you are tolerant to index date misclassification - use the H/O code. If you are not tolerant and dont use H/O code, then you have sensitivity errors.
I think the usage of exposure vs outcome vs feature cohort came when there was discussions that certain types of errors are ok for exposure cohorts, but not ok for outcome cohorts. e.g. i have heard arguments that sensitivity errors are less problematic for exposure cohorts but more problematic for outcome cohorts.
Does anyone know if these ideas have been clearly articulated and, hopefully, empirically tested? If not, then promoting those ideas (i.e. feature cohort vs outcome cohort vs exposure cohort) may not be valid.
These clearly warrant different phenotypes. Those where we care about the specificity (yea I used the word to be confusing!) of the index date and those where we do not. If I need a phenotype to define a baseline condition either for exclusion (or even establishing inclusion/exclusion criteria for a trial screening) or confounding adjustment I don’t care when you had it prior to some index exposure date of interest so as soon as my data knows this condition exists it’s good enough for me (US data sources for chronic conditions will endlessly diagnose me with this condition in annual 365d increments, UK/NHS/EMR systems might record this and move on). If entering this diseased cohort is my exposure of interest then it is my index date and I’m quite sensitive (again yea I choose the word to be confusing!) to the index date. Same for outcome where I do really need to know what date you officially meet the phenotype.
When I change insurance carriers at open enrollment my first phone call is to member services to record that my appendix has been removed so that any observational study conducted within the resource can appropriately remove me from the at risk population for appendicitis which is why you’ll see a Z90.89 “Acquired absence of other organs” with a free text comment ‘appendix’. I’ll probably instruct them to record it as a telehealth visit to make it look like I took out my appendix over the phone.
Short of complete data linkage from birth to Medicare and death. I think there is value in the nuggets of information left by H/O codes or what appear to be train wreck patients in the first six months within a GP office but are really just real world data collecting a patient history.
I have heard it well-articulated from You-Know-Who. Tested? No. We have a problem with phenotype evaluation as you know Should do more stuff with Joel.
In your example, the history of code would be useful for December 2010 prevalence, bad for incidence. For incidence we need precise start date. For prevalence we need to know if it started prior to and person still has the phenotype.
The only piece of data on this thread (characterization proposed above) shows that by dismissing ‘H/O’ codes you will be potentially underestimating prevalence of a feature (by not counting thousands of people with a history of disease) as well as overestimating incidence of a target (by counting as new events that were already known/prevalent) in many data sources (European primary care data), whilst not in others.
Of course I care about dates, that is precisely why I want to use ‘H/O’ codes differently depending on the use of a cohort/phenotype as a ‘feature’ vs ‘outcome’. Your solution to simply not use these codes results in the dismissal of thousands of cases, which leads to systematic bias due to incompleteness of prevalent and overestimation of incident events.
This has nothing to do with how OHDSI constructs a cohort/phenotype. Let us not forget people have been creating cohorts from RWD for a while, also out of the OHDSI community. And this ‘issue’ is well known to many of us who use European primary care data for research, both in and beyond OHDSI.
So based on published data (thanks @Kevin_Haynes for the reference), internal data, and the characterisation above, it is clear that having phenotype flavours is useful to minimise incompleteness if you care about having a fuller picture of prevalence and/or incidence. Of course, this does not apply to all study types, but it does apply to a lot of the work we are currently doing in my team.
The example of estimating prevalence and ‘H/O’ codes is just one use example as I said in my first post. Other ‘flavours’ of phenotypes will be needed if one is to reuse these phenotypes for future studies. Target, outcome, feature flavours is a good starting point
No, I did not say dismiss it. I am saying when we use it we will not know that incident date precisely. It causes an error.
If your use case is prevalence and the study period is after the date the code was used - then it’s fine. If your use case is prevalence before that then we have a problem of understanding prevalence.
Example: if the history of code was found April 1st 2000, does that mean the person has the prevalent disease on February 2000.? How about prevalence calculation on March 1999?
In reality the person may have had migraine since 1995, then they changed systems, a new system captured the history code on April 1st 2000
that’s all it is. If the history of codes are found to be used AFTER the period you calculated prevalence for - then you have a source of error.
I think this reads to me like “do not use”. so yes, you said to dismiss it
Now, on this other topic:
I cannot think of any scenario where I would use any code (h/o or otherwise) recorded AFTER the period we calculate prevalence. that would no longer be a prevalence. that is true for all and any codes, irrespective of their ‘h/o’ nature. So I am not worried because this is not a problem for prevalence analyses if that calculation is done correctly, i.e. using data from the time on or before the date when the prevalence is obtained on
talking about error: what about the issue with underestimating prevalence and potentially overestimating prevalence observed when ‘dismissing’ the “H/O” code for migraine seen in the characterisation discussed above? this DOES worry me
So - i am still thinking that we should not use H/O code in entry event criteria because it is causes imprecise index date (cohort start date). In most data sources, like you said, these codes are practically non existent - so they did not matter. You are bringing up a scenario where the codes are being used in sufficient numbers that it matters. So, we have to decide how to handle them. If we use them - we get imprecise start date, if we dont use them we have lower sensitivity.
Errors: there is errors in everything we do. The science is to reduce these errors as much as we can by identifying them and trying to reduce them by picking the error that causes the least bias.
