🏥 Medical Entity Extraction from Clinical Notes

Extracting medical entities — Problems, Treatments, and Tests — from unstructured clinical notes using Differentiable Neural Computers (DNC) and MIMIC-III pretrained BERT.

---

📌 Overview

Clinical notes contain rich but unstructured medical information. This project applies Named Entity Recognition (NER) to automatically identify and classify medical entities from clinical text — a critical step in building intelligent healthcare AI systems.

This work is directly relevant to real-world Healthcare AI applications including:

• EHR data processing and structuring
• Clinical decision support systems
• Patient readmission risk modeling
• Medical coding automation (ICD-10, SNOMED CT)

---

🎯 Problem Statement

Given unstructured clinical text like:

"Patient was diagnosed with pneumonia and prescribed amoxicillin. A chest X-ray was ordered."

The model extracts and classifies:

Entity	Type
pneumonia	Problem
amoxicillin	Treatment
chest X-ray	Test

---

🧠 Approaches Used

1️⃣ Differentiable Neural Computer (DNC)

• Adapted from the original DNC paper (Graves et al., 2016)
• Treats entity extraction as token classification
• Uses BIO tagging convention
• Word embeddings from BioWordVec (PubMed + MeSH pretrained)

2️⃣ MIMIC-BERT Token Classifier ⭐ (Best Results)

• Fine-tuned BERT pretrained on MIMIC-III clinical notes
• Used HuggingFace Transformers for fine-tuning
• BertForTokenClassification with Adam Optimizer
• Significantly outperforms DNC on all entity types

---

📊 Results

DNC Model

Entity Type	Precision	Recall	F1 Score
Problem	0.78	0.74	0.76
Test	0.85	0.62	0.72
Treatment	0.83	0.62	0.71

• Overall Accuracy: 66.76%
• Macro F1: 0.73

MIMIC-BERT Model ✅

Entity Type	Precision	Recall	F1 Score
Problem	0.84	0.85	0.85
Test	0.84	0.90	0.87
Treatment	0.87	0.88	0.87

• Overall Accuracy: 99.78%
• Macro F1: 0.87

---

🛠️ Tech Stack

Category	Tools
Deep Learning	PyTorch, PyTorch Lightning
NLP / Transformers	HuggingFace Transformers, BERT (MIMIC-III)
Word Embeddings	BioWordVec (PubMed + MeSH)
Data Processing	BertTokenizer, BIO Tagging, word2vec
Dataset	2010 i2b2/VA Challenge Clinical Notes

---

📁 Project Structure

medical-entity-extraction-nlp/
│
├── data/                   # i2b2/VA dataset (not included - see below)
├── models/
│   ├── dnc/                # Differentiable Neural Computer implementation
│   └── bert/               # MIMIC-BERT fine-tuning scripts
├── preprocessing/          # Data cleaning & BIO tagging
├── results/                # Model evaluation outputs
└── README.md

---

🚀 Getting Started

Prerequisites

pip install torch transformers scikit-learn numpy pandas

Dataset

This project uses the 2010 i2b2/VA Challenge dataset. You can request access at: 👉 https://www.i2b2.org/NLP/DataSets/

Run BERT Model

python models/bert/train.py \
  --data_dir ./data \
  --model_name_or_path emilyalsentzer/Bio_ClinicalBERT \
  --num_train_epochs 5 \
  --batch_size 16

---

🔬 Key Learnings & Connection to Real-World Healthcare AI

This project directly mirrors challenges I worked on at Capgemini building clinical ML systems:

• NLP pipelines for extracting entities from unstructured EHR data
• Token classification using transformer-based models (similar to BERT fine-tuning)
• Healthcare data standards — aligning outputs with ICD-10 / SNOMED CT codes
• Model evaluation with precision/recall/F1 across medical entity classes

---

📄 References

1. Graves et al. Hybrid Computing Using a Neural Network with Dynamic External Memory. Nature 538, 2016.
2. Franke et al. Robust and Scalable Differentiable Neural Computer for Question Answering. arXiv:1807.02658, 2018.
3. Zhang et al. BioWordVec, Improving biomedical word embeddings with subword information and MeSH. Scientific Data, 2019.
4. Uzuner et al. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. JAMIA, 2011.
5. Peng et al. Transfer Learning in Biomedical NLP. arXiv:1906.05474, 2019.

---

🤝 Connect With Me

---

"Building AI that's not just powerful, but trustworthy."