OHDSI Home | Forums | Wiki | Github

Creating HEDIS measures as cohort definitions in ATLAS

Researchers
#1

@Gowtham_Rao, @bnhamlin, @Daniella_Meeker and whoever else may be interested in applying OHDSI to use cases around quality improvement:

As a quick demonstration, I created two instances of the HEDIS measure of ‘BMI Assessment’ (ABA) in ATLAS:

  1. Using standard concepts (which provides a much more efficient manner of defining the value sets, and seems to sweep in extra ICD9/10 codes that should have been captured but are apparently missed): http://www.ohdsi.org/web/atlas/#/cohortdefinition/87847

  2. Using source concepts that define the valuesets for BMI and BMI percentile, just to demonstrate how ATLAS can create cohorts using source concepts instead of standards (even though, as a general principle we discourage this behavior): http://www.ohdsi.org/web/atlas/#/cohortdefinition/87846

I’ll post to this thread as I go through and do more. As I read the measures, I haven’t yet encountered anything I think can’t be directly supported within ATLAS given all of the enhancements that @Chris_Knoll has made to the standard cohort definition module. There’s a minor bug around observation_period that appears to have been introduced in the last release, but I expect Chris will have a hoxfix for that shortly. I’ll be curious if others tried to use ATLAS and have found specific limitations, in which case, please post to this thread so we can learn.

1 Like
Making a QI (and health system executive) Case for OHDSI
#2

Here’s the HEDIS measure for 'Chlamydia screening in women (CHL) in ATLAS:
http://www.ohdsi.org/web/atlas/#/cohortdefinition/87848

This has been a useful learning experience for me. While I’ve always been
a fan and proponent of the use of standardized vocabularies, I think going
through these HEDIS valuesets is a clear demonstration of why using medical
ontologies as an analytical structure is an essential requirement of any
standard body, because lists of codes from controlled terminologies is
unwieldy and will often be incomplete. Instead of people worrying about
‘what information will be lost’ when converting data to OMOP CDM, people
should open their eyes to ‘what information will be gained’, and a large
part of that is being able to leverage the ancestral relationships from
standard vocabularies like SNOMED, ATC, LOINC to create succinct and
expressive conceptsets that can represent extraordinarily large set of
source codes.

#3

One more for good measure (pun intended…groan):

Here’s a draft HEDIS measure for 'Use of Imaging Studies for Low Back Pain
(LBP) in ATLAS: http://www.ohdsi.org/web/atlas/#/cohortdefinition/87849

This one had many more exclusion criteria, but ATLAS handled those well.

Bottom line from this little experiment: I know little-to-nothing about
HEDIS and quality measure development, but I was able to directly translate
the HEDIS description into a computer-executable cohort definition (both
numerator and denominator) in ATLAS for 3 different measures in about 3
hours. All 3 measures ran within my local environment against 3 large US
claims datasets, and each completed in < 5 minutes, producing the numerator
and denominator counts, and the results proportion that is the primary
output metric.

If I’m reading the documents correctly, there’s only about 50 HEDIS
measures out there now, so that means if folks in the OHDSI community are
so inclined, if a handful of people look a couple days each, then the whole
community could benefit from an open-source library of HEDIS phenotypes and
could directly produce quality report metrics. This could be a great
opportunity for collaboration for those with common interest…raise your
hand on this thread:)

#4

I think creating libraries to be able to compute “standard measures” on healthcare data would be a great value add to the OHDSI community. Maybe eventually we can try to get the library officially certified: http://www.ncqa.org/tabid/217/Default.aspx

1 Like
#5

As somebody who thinks that the vocabulary work is tremendous and useful, I do have to caution that there is often a method to the madness of people building quality measures. If I am being evaluated based on achieving a metric, I want to have the most specific definition possible (i.e., specificity close to 100% and a positive predictive value close to 100%). I don’t want to include false positives because they are more likely to not achieve the measure. If my sensitivity is low, I am ok with that – I want to be evaluated on clearly defined cases.

I mention this because the inclusion or exclusion of codes affects the sensitivity and specificity of the algorithm used to identify the event of interest. One of the benefits of automated systems like ATLAS is that they can be used to define things in multiple ways, easily, to inform this kind of discussion.

#6

Some thoughts- In addition to the work we did with Laura on HQMF, Ayan has implemented a number of these, leveraging the value set authority center, which should give you the correct codes and I believe has a working API now so latest are available. Like & EMERGE network have made a very nice authoring tool that generates HQMF also integrated with VSAC.

I have been incubating some ideas on FHIR and CQF/CQL, which is much better than HQMF for computability, but the standard is not fully balloted.

#7

HEDIS measures are based on Anchor index date like your post at Index event selection based on calendar date · Issue #322 · OHDSI/Atlas · GitHub

Adult BMI Assessment:

Age calculation for Adult BMI assessment is anchored to December 31st of measurement year;

  • person should be atleast 18 yrs as of Jan 31st of the year prior to
    the measurement year (i.e. should be >= 20 yrs on Dec 31st of
    measurement year)
  • person should be less than 74 years as of Dec 31st of measurement year

@Patrick_Ryan - I think you have configured it for age to be calculated on the earliest outpatient-visit: which is not exactly the same because, some people may become 18 after Jan 31st of the year prior to measurement year; or may already become 74 on date of outpatient visit etc. Without the index date we cannot achieve what the measure is asking for - but we have an approximate. HEDIS measures needs to be precise - unfortunately.

Continuous Enrollment: This measure requires continuous enrollment for two consecutive years - both the measurement year and prior year. I think you are trying to achieve that by “with continuous observation of at least 365 days before and 365 days after event index date” But the measure rule also allows for ONE 45-day gap dung each year. There is another way to solve it; by using Observation Period and checking to see if the person had atleast (365 days - 45 days) observation days AND not more than 2 occurrences of observation period. I will post a screen-shot - waiting for the bug to be corrected.

Pasted image696×358 29.1 KB

#8

As I read the measures, I haven’t yet encountered anything I think can’t be directly supported within ATLAS given all of the enhancements that @Chris_Knoll has made to the standard cohort definition module.

I am not familiar with the latest changes by Chris. But to facilitate identifying potential “things that can’t be directly supported within ALTAS” I want to point out some sophisticated aspects of certain HEDIS rules. Representing these may require changes to ATLAS and/or the data model.

Utilization and risk adjusted utilization

HEDIS defines utilization using “member months” as the denominator (e.g. Visits / 1000 member months). Calculation of member months is sophisticated, and depends on administrative data reflecting membership of patients (I am not sure if it is contained in the OHDSI data model); without it, several utilization metrics in HEDIS cannot be correctly calculated.

Resource use

The resource use metrics require “standard price” and “standard cost” values to be available for various services. Factors such as payer liability, member cost sharing, etc. can complicate their calculation. My understanding is that an “average” number is often used, based on some cost algorithm, and the number is updated on some frequency. Do we have a way to store and use cost/price data in ATLAS?

Denied claims

Take the NCS metric in the “Overuse/Appropriateness” group as an example. The metric requires excluding denied claims. Does the data model have a way to represent which claims were denied? Does ATLAS have a way to account for it?

#9

@salmasian

You bring up some good points and these are very important. Scope of ATLAS we are looking into right-now is what is called a ‘Cohort-builder’, i.e. it selects a list of individuals who meet certain inclusion and exclusion criteria. These criteria may be based on demographics, procedure, conditions, visits etc.

Once a cohort has been built - then we can derive/calculated attributes to every individual in that cohort, such as ‘member months’/exposure; or (standard) costs. These are important information required to solve the full-scope of HEDIS needs, but the first step is cohort-builder.

At high-level, the steps would be: (We are at step #1 for now. )

  1. Cohort builder - identify list of person_id’s that meet the inclusion/exclusion criteria
  2. Calculate the attributes of interest for each person in the cohort (member months, costs, services, utilization etc)
  3. Use for downstream calculation, tabulation or visualization: % of individuals with low back pain that had an imaging.

Are there any measures you want to try out - we could prioritize and solve 5 to 10 measures in Atlas? My vote for the top 5 are (ofcourse, each has subsets - and so the actual list is very long):

  1. Chlamydia Screening in Women
  2. Use of Imaging Studies for Low Back Pain
  3. Controlling High Blood Pressure
  4. Immunizations for Adolescent
  5. Comprehensive Diabetes Care (Retinopathy)
#10

There are so many opportunities for the ATLAS cohort builder to assist in measure development as well as measure reporting. The limiting factor in electronic measurement in HEDIS is an ability to electronically specify “eligibility” for any particular measure. In traditional ( e.g. admin claims) HEDIS we use a combination of continuous enrollment within a certain payer as well as encounters within the measurement period. This work (sort of) when limited to claims and payer eligibility files, however the new generation of measures expected in HEDIS 2018 use only the enrollment information to define a cohort for measurement. I expect that the screening measures that currently exist in our EOC domain will likely follow suit, therefore I would love to see and active community debate on layout for attribution.

I would be happy to provide whatever other details that might help the community come to some consensus on building this type of “complex” cohort in a highly standardized fashion. The HL7 workgroup is currently working with us to define but these conversations have just started so now is a perfect opportunity.

P.S. I would vote for Adolescent Immunizations, Pneumococcal Status for older adults, any of the Depression/Chlamydia/Breast Cancer/Unhealthy Alcohol Use Screening measures. I would stay as far away from CBP as possible since there are several version currently in use and the JNC guideline update is expected over the summer which may change the spec considerably (or result in the measure’s retirement)

#11

The data collection for HEDIS Relative Resource Use measures have just been stopped. The measure specifications are still valid, however the data collection and reporting burden were deemed to high for the value of the information that the measures were providing.

#12

The VSAC does not use HEDIS definitions, only those approved for eMeasures stewarded by NCQA. We have all HEDIS value sets as a FHIR resource if you are interested.

We are specifying a select set of HEDIS measures using the CQL-HQMF format as well as convening a tooling workgroup to get these and the QDM-HQMF version to CQL-FHIR (eventually).

#13

The bug with selecting Observation Period criteria has been fixed on ohdsi.org. Version 2.0 of WebAPI will support specifying a observation period start/end date override.

#14

Awesome! Thank you version 2.0

#15

Friends:

Apart from the API, do you guys still need changes to the CDM to cover Healthcare Quality Measures? Daniella has a proposal in for that. Let’s use the upcoming face-to-face for closing out a lot of these.

#16

HEDIS measures are based on Anchor date. @Patrick_Ryan @Chris_Knoll is it possible to do the following in Atlas v2.0

Want to identify cohort of people who are between 50 to 70 years on December 31st 2010

  • Anchor date December 31st 2010. (an observation period: period start is on or Before 2010-12-31, period end is on or after 2010-12-31)
  • Age between age of 50 to 70 as of December 31st 2010: I dont think we can meet this criteria yet. Age computation options are “with age at period start between 50 and 70 (inclusive)” not “with age on 2010-12-31 between 50 and 70 (inclusive)” Would this need an enhancement to Atlas? @anthonysena

See attempt here: http://www.ohdsi.org/web/atlas/#/cohortdefinition/98077

#17

Yes, you can do that now with Atlas 2.0. I’ve made a copy of your definition with the modifications:
http://www.ohdsi.org/web/atlas/#/cohortdefinition/98078

You set your primary event to look for observation periods (as you have done) except you use the option ‘Specify start and end dates’. I set this to specify a 2010-12-31 start and 2010-12-31 end date. Next, I added an ‘Initial event inclusion criteria’ to further restrict the primary events where the person must be between 50 and 70 as of the primary event’s start date (2010-12-31).

Note, when you specify a start and end date, only observation periods that ‘cover’ those dates are returned (ie: observation period start date <= user specified start date and observation period end date >= user specified end date.

#18

Good morning

I think the community may benefit from a workgroup that creates and maintains quality measures.

Thoughts?

#19

I agree. When I first found out about OHDSI and the CDM I thought of the QDM that NQF has developed. They define eCQM in a domain specific language to reduce any ambiguity in the measure definitions. I believe NCQA uses the same measure definitions as well. One interesting opportunity would be to use the measure definition DSL to generate JSON for ATLAS.

FR

#20

I agree.

t