Heritage Damage Detection

Korea's cultural-heritage survey workflow was paper-based: inspectors recorded site condition and damage by hand, with no structured way to track designation status, ownership, or damage severity over time, and no shared real-time view across inspectors in the field.

The system needed to work across Web, Android, and iOS, integrate an XML-based public heritage API, support multiple field inspectors, and preserve an auditable edit history. The model needed to identify four documented deterioration types from field photographs and return a severity grade that inspectors could review.

I designed and implemented the object-detection system. Its PyTorch pipeline uses DETR and CLIP to locate damage in field photographs, classify the deterioration type, and attach a confidence score. Inspectors can review the output through the Flutter field app and FastAPI backend. I completed this work with the Natural Language Learning Lab and Korea Heritage Service; the deployment and jointly filed patent are team outcomes.

• One field client: Flutter provided a shared codebase for Web, Android, and iOS.
• API normalization: A FastAPI proxy converted the public heritage API's XML into JSON and supported multi-criteria search.
• Reviewable inference: The detection pipeline (DETR, CLIP) returns bounding boxes, class labels, confidence scores, and A–D severity grades instead of an opaque single label.
• Auditability: Firestore streams synchronized records in real time while automatically recording edit history.

The inspector workflow is organized into Field Survey, Inspector Notes, and Comprehensive Assessment. The pipeline checks on-site photographs for four deterioration classes: cracks, splits, biological decay, and crushing or burst damage. Survey metadata and edits synchronize across clients.

• Model quality: F1 0.89 (detection model) across four deterioration types
• Inference latency: ~800–1200ms (CPU), ~200–400ms (GPU) per image
• API response: <150ms (heritage search), <2s (AI detection)
• Sync delay: <500ms (Firestore real‑time updates)
• Model: DETR (ResNet‑50), 4 classes, class‑specific conf 0.15–0.30, NMS IoU 0.1, ~552MB
• Coverage: Single codebase → Web, Android, iOS; tested 10+ concurrent users

The system is in operational use at Korea Heritage Service. A patent covering the damage-detection method was filed jointly by the Natural Language Learning Lab at Jeonbuk National University and Korea Heritage Service, and the software is registered for copyright.

The evaluation covered four documented deterioration types. Keeping the inspector in the review flow is essential when model output informs field assessment.