Visualizations for Interpretability

[ishkavi]

Hi,

I recently came across your paper when I was looking for some multi-label classification techniques. Yours is a very interesting work, and thank you very much for making your code publicly available. A major reason I am interested in this work is the claim of interpretability. I know it has been some time since this code was written, but I have a question regarding that and it would be great if you could give some insights.

Do you remember how you generated the 3 visualisations mentioned in the paper? I noticed some configurations such as int_preds, and attns_loss in the paper. However, I am not sure how you exactly did that and it would be great to get some insights on that.

[jacklanchantin]

Hi, this can be visualized using the attn_output_weights that are returned from nn.MultiheadAttention:

multihead_attn = nn.MultiheadAttention(embed_dim, num_heads)
attn_output, attn_output_weights = multihead_attn(query, key, value)

[ishkavi]

Thank you very much for the swift response. I have a couple of follow-up questions.

1. I am still not clear on how you differentiate Feature to label attention and Label to label attention weights.

2. I am using a dataset with numerical features (E.g. Every sample has X number of numerical features). Although the model is working fine with this dataset, it creates a dictionary for every single numerical value (similar to using a bag of words). As a result, I believe that the Intermediate predictions visualization and Label to Feature attention weight visualization don't provide much meaning in my context. Do you think there is a different way of handling this situation (using these visualizations in the context I described)?

[1074051286]

Thank you very much for the swift response. I have a couple of follow-up questions.

1. I am still not clear on how you differentiate Feature to label attention and Label to label attention weights.
2. I am using a dataset with numerical features (E.g. Every sample has X number of numerical features). Although the model is working fine with this dataset, it creates a dictionary for every single numerical value (similar to using a bag of words). As a result, I believe that the Intermediate predictions visualization and Label to Feature attention weight visualization don't provide much meaning in my context. Do you think there is a different way of handling this situation (using these visualizations in the context I described)?

hi,i get the same question with you. Do U solve the question? I cant find the nn.MultiheadAttention mentioned above,Could U plz give me some help? thx