top of page
Search

The NOMA AI App: From Python Code to Clinical Device

  • Writer: Anie Etor-Udofia
    Anie Etor-Udofia
  • Apr 1
  • 3 min read

The User Experience Flow

When a user launches NOMA AI, here's what happens:

  1. Camera Feed: Live preview of the skin lesion (640×480 RGB)

  2. Capture & Analyze: User positions lesion in center, taps the large button

  3. AI Inference: Model processes the 224×224 image (0.3-0.5 seconds on Pi 4)

  4. Clinical Assessment: 7-step ABCDE wizard with swipe navigation

  5. Risk Fusion: Combines AI + ABCDE + patient factors

  6. Result Display: Diagnosis, recommendations, and Grad-CAM heatmap

  7. LED Feedback: Red/Yellow/Green physical indicator

  8. Health Passport: Assessment saved with timestamp


Critical Design Decisions

Why Yellow LED Blinks During Analysis?Users need immediate feedback that the system is working. The yellow blinking pattern (500ms intervals for up to 10 seconds) indicates processing without making them stare at a frozen screen.

Why Swipe Navigation?The clinical assessment wizard has 7 steps. Swipe gestures (with a 100px threshold) let users navigate naturally while preserving touch targets for buttons. The SwipeableWidget class checks whether the touch started on an interactive element (radio button, checkbox) before triggering navigation.

Why Grad-CAM Heatmaps?Trust is critical in medical AI. Grad-CAM highlights which image regions influenced the diagnosis. Red areas = high influence, blue = low influence. This turns the AI from a "black box" into an explainable tool.


Handling Camera Color Formats

The Pi camera can output various formats. The code automatically detects and corrects:

python

# Sample center pixel to detect color order
h, w, _ = frame.shape
sample = frame[h//2, w//2]
if sample[2] > sample[0] + 30:  # Blue > Red by threshold
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

This ensures colors display correctly on the touchscreen.


GPIO LED Control with Fallback

The SimpleLED class handles both real hardware and mock mode:

python

def set_led(self, color, state):
    if self.gpio_available:
        self.GPIO.output(self.led_pins[color], self.GPIO.HIGH if state else self.GPIO.LOW)
    else:
        print(f"Mock LED {color}: {'ON' if state else 'OFF'}")

This allows development without hardware and prevents crashes if GPIO fails.


The Clinical Assessment Wizard

The 7-step wizard walks users through ABCDE criteria:

  1. A - Asymmetry: Is the lesion asymmetrical?

  2. B - Border: Irregular, ragged, or notched?

  3. C - Color: Single color, 2-3 colors, or many?

  4. D - Diameter: Small (<6mm), Medium (6-10mm), Large (>10mm)

  5. E - Evolution: No change, slow change, rapid change?

  6. Patient Information: Age, skin type, risk factors, clinical context

  7. Summary: Review all answers before calculation

Each step includes:

  • Visual diagrams (drawn with QPainter)

  • Clear descriptions

  • Large touch targets (radio buttons with 20px text)

  • Scroll areas for longer content


Risk Calculation Formula

The final risk score combines three components:

text

Total Risk = (ABCDE_Score / 8.0) × 40
           + (Patient_Score / 4.0) × 20
           + (CNN_Weight × Confidence) × 40

Where:

  • ABCDE_Score: 0-8 points based on responses

  • Patient_Score: Age (>50 = 1), Skin Type (I/II = 1), Family History (1), Sunburn History (1)

  • CNN_Weight: 3 for malignant, 1 for benign, 0 for normal

LED Color Mapping:

  • Red (≥70): High risk - Urgent dermatology referral

  • Yellow (40-69): Moderate risk - Schedule follow-up within 4-6 weeks

  • Green (<40): Low risk - Continue self-checks


Health Passport: Longitudinal Tracking

Every assessment saves to health_passport.json:

json

{
  "timestamp": "2026-03-28T14:30:00",
  "ai_prediction": "Melanoma",
  "confidence": 0.87,
  "abcde_score": 6,
  "patient_score": 3,
  "total_risk": 72.5,
  "led_color": "RED",
  "image_path": null
}

This enables trend analysis—users can see if risk scores are improving or worsening over time.


Educational Tip Banner

After each analysis, a random dermatology fact appears:

text

💡 Did you know? Sunscreen with SPF 30+ reduces skin cancer risk by 40-50%.

The banner auto-hides after 10 seconds, reinforcing education without cluttering the interface.


Error Handling and Resilience

The app includes several layers of error protection:

  1. Global Exception Handler: All crashes logged to crash_log.txt for debugging

  2. Camera Fallback: If camera fails, dummy frames keep the app running

  3. GPIO Fallback: Mock LED mode if hardware unavailable

  4. Low Confidence Warning: If confidence < 60% or uncertainty > 80%, prompts retake

  5. Service Auto-Restart: Systemd restarts the app if it crashes

Comments

bottom of page