OHDSI Home | Forums | Wiki | Github

LAERTES: Explaining modality

Developers
#1

A question came up about what "modality"means in the LAERTES evidence base.

First note that LAERTES consists of an evidence base (which contains data that can be used via some process to arrive at knowledge of a drug-HOI association) and an (as not yet created) knowledge base. Modality is currently only relevant to the evidence base

Each piece of evidence, identified by a drug-health outcomes of interest (HOI) pair and evidence type (e.g., case report, clinical study, product label mention, etc), has a modality indicates if this pair was a positive association (i.e. the drug and condition are considered an adverse reaction together) or negative association (i.e. it is known this drug does not cause this condition). SemMedDB is the only source with explicitly negative associations (e.g., see the “exact” column in the results of this query).

“The hyperventilatory response to hypoxia was not altered by propranolol.”
“Most of those with pre-treatment abnormalities of liver function developed abnormalities in their biopsies, not attributable to alcohol.”
etc…

The other datasources are assigned a positive modality because the presence of the evidence in the source means implicitly supports drug-HOI association (e.g., an HOI in a drug’s table of ADRs) or, the source does not provide a way to distinguish positive and negative evidence (e.g., MeSH tags indicate that a title and abstract is about a drug-HOI but there is no other information).

Clearly, this is up for discussion and hence this forum topic…

LAERTES Team Meeting Agenda
#2

Two thoughts that come to mind. Let me describe them as best as I can:

  1. For drug-HOI associations that we know are positive and negative (or no association, for purposes of semantics), is there a way of characterizing this information without having to disclose whether it’s a positive or negative association? for example, can we just characterize the presence and absence of a drug-HOI relationship as presence = t and presence = f, respectively?

  2. Is there any evidence in LAERTES, incorporated from many sources, that hasn’t been labeled as a positive or negative (no) association? that is, modality = null?

#3

I would argue the lack of evidence of something is different than there is evidence there is no association. My understanding is that SemMedDB is finding evidence that there is no association. Lack of evidence means that there was no positive associated evidence suggesting otherwise and then the assumption is the lack of positive associated evidence means that there is no association.

@rkboyce let me know if I’m wrong here!!

No, the modality column is never null.

SELECT *
FROM DRUG_HOI_EVIDENCE
WHERE MODALITY IS NULL
#4

Erica - can I point out a contradiction in your statement?

“I would argue the lack of evidence of something is different than there is evidence there is no association…Lack of evidence means that there was no positive associated evidence suggesting otherwise…”

Okay, so lack of evidence = no positive association but not entirely a negative (or no) association

“…and then the assumption is the lack of positive associated evidence means that there is no association.”

Aren’t you saying that a lack of evidence then assumes no association? You argued in your previous sentence than the two are different.

I’m just nitpicking because it’s important to seek universal clarity on this issue, as there was confusion last week’s community call.

#5

I would agree with Erica that the lack of evidence of something is
different than there is evidence there is no association. Also, I don’t
see a contradiction in…

“…and then the assumption is the lack of positive associated evidence
means that there is no association.”

The key word here that avoid contradiction is “assumption”. By making
this assumption we go into the realm of inductive reasoning (see
https://en.wikipedia.org/wiki/Problem_of_induction) . With negative
modality evidence, (in principle) there no need for inductive reasoning
based on the lack of positive modality evidence because explicitly
negative evidence exists. Examples can be seen using the query posted
earlier in this thread.

As for Lili’s questions:

“1) For drug-HOI associations that we know are positive and negative (or
no association, for purposes of semantics), is there a way of
characterizing this information without having to disclose whether it’s
a positive or negative association? for example, can we just
characterize the presence and absence of a drug-HOI relationship as
presence = t and presence = f, respectively?”

Would you please explain why you think this is better than the approach
we are currently using which is simply explicitly tagging the epistemic
modality of an evidence item? It is not that a change is not warranted,
but please make the case for making a change more justified.

“2) Is there any evidence in LAERTES, incorporated from many sources,
that hasn’t been labeled as a positive or negative (no) association?
that is, modality = null?”

Erica already answered this.

#6

  1. Regarding “assumption” and inductive reasoning, you and Erica are saying that ‘no association’ between a drug an HOI is not the same thing as ‘lack of evidence’ between a drug an HOI. But because you choose to make the assumption that a lack of positive associated evidence, then there is no association between the drug and HOI. Right?

  2. [quote=“rkboyce, post:5, topic:1074”]
    Would you please explain why you think this is better than the approach
    we are currently using which is simply explicitly tagging the epistemic
    modality of an evidence item? It is not that a change is not warranted,
    but please make the case for making a change more justified.
    [/quote]

Not sure what you mean by ‘epistemic’ modality, but remember our discussion with Manfred from last week’s call that prompted changing ‘negative association’ to ‘no association’. We were going over the following table:

Association OMOP Ref Set LAERTES Dataset
Yes GROUND_TRUTH = 1 modality = true
No GROUND_TRUTH= 0 modality = false
Lack of evidence No evidence No evidence

How do we account for the 4th row (no evidence)? Hence my suggestion for creating a ‘presence’ to account for this time of information, to account for the presence of absence of information. Just as you and Erica would agree, “lack of evidence of something is different than there is evidence there is no association,” unless you make the assumption mentioned above. Get my gist?

I’m just making a suggestion in response to people’s comments, so as to avoid confusion. That’s all :).

#7

Just thinking about this topic some more after my last post. I find it interesting that one has to make the assumption and use inductive reasoning to interpret that a lack of positive associated evidence as equivalent to no association.

If you want LAERTES to be widely adopted by drug safety practitioners (which is your vision right?) you need to make it user-friendly as possible. Requiring a user to know how to make an assumption and use inductive reasoning is an additional burden you are expecting from your audience. Take the LAERTES community calls - it took me and others a few conversations to iron out this concept of inductive reasoning. By labeling a type of evidence as ‘absence’ (presence = false) you can eliminate that burden from your audience, enabling them to better understand the data with less effort.

Once again, I’m just offering my perspective with the intent of improving the LAERTES data format. Take it for what it’s worth.

#8

I think that the explanation motivating the suggested change is much
more clear than when you first posted it so that I now understand what
you are requesting and think it makes sense. Thanks for posting the
results of our last conversation. That helps to clear up allot. There
will be a change to make it easier to understand.

t