(Enhanced by Generative AI)

Njawani is an Android app that uses real-time object detection to make learning aksara Jawa (Javanese script) interactive and fun.

A. The Problem: A Script Fading Away

Javanese script — Aksara Jawa — is one of Indonesia's traditional writing systems, used for centuries to record everything from literature to everyday notes. But if you walk through the streets of Java today, you'll be hard-pressed to find it. Modernization has pushed it out of daily life. Transliterated into Latin letters, it survives mostly in textbooks and elementary school curricula — a mandatory local subject that students often find monotonous.

The real problem isn't that young people don't care about their heritage. It's that the way they're being taught hasn't evolved. Children today grow up with smartphones in their hands, yet the tools for learning their own culture remain stuck in the past. When conventional teaching methods feel like a chore, it's no surprise that interest in Javanese script continues to decline.

B. The Search for a Solution

The idea started with a simple observation: people already spend hours looking at their screens. What if, instead of fighting that reality, we could use it as a bridge to cultural learning?

The concept took shape around a familiar interaction — pointing a camera at something. Every smartphone user already does this instinctively. What if pointing your camera at an everyday object could also show you how to write that word in Javanese script?

C. Building the Foundation

Before writing a single line of code, the development needed a clear roadmap. The ADDIE framework provided structured five phase process that would keep the project grounded in user needs rather than technical assumptions.

User requirements identified:

1. Real-time object detection using smartphone camera

2. Display detected object's Javanese script writing

3. Capture and save detected images to gallery

4. Browse and manage saved images

The Design phase translated those needs into a user flow:

1. Splash screen — App launch, permission checks

2. Main screen — Welcome ("Sugeng Rawuh!"), three navigation choices

3. Camera screen — Live detection with bounding box and script output

4. Preview screen — Review and save captured images

5. Gallery screen — Browse, view, delete saved images

D. The Machine Learning Challenge

The model needed to be accurate enough to be useful but lightweight enough to run on everyday smartphones. I actually planned to making my own dataset, the dataset that fit in indonesian urban life, but unfortunatelly limited by time and resource so for MVP i just use dataset that already on the internet, even though some classes are less likely to find in indonesia.

Key decisions at a glance:

1. Model: SSD MobileNet V2 — mobile-optimized single-shot detector

2. Input size: 384×384 — reduced from 640×640 for faster processing

3. Max detections: 1 per frame — focused, simple user experience

4. Dataset: COCO 2017 — 80 categories of everyday objects

5. Format: TensorFlow Lite with post-training quantization

After retraining, the model achieved a mean Average Precision (mAP) of 28.2 on the validation set — more than adequate for an educational app where the goal is exposure and engagement.

The final step was converting the model to TensorFlow Lite format with post-training quantization — a technique that shrinks the model size, improves inference speed, and enables GPU acceleration on supported devices.

E. Bringing It All Together

The app was built in Android Studio using Java, with OpenCV handling camera integration.

How the detection works:

1. Camera feed captures each frame in real time

2. Frame is resized to 384×384 using scaling method (preserves full image)

3. Convolutional layers with 3×3×3 kernels extract features from RGB channels

4. Model identifies objects and returns label + bounding box

5. Javanese script for the detected object is displayed on screen

When an object is recognized, the app draws a bounding box around it and displays the corresponding Javanese script. If nothing is detected, a simple message — "Tidak ada objek dikenal" — tells the user to try a different object.

F. The Moment of Truth

Object detection testing — Pointed the camera at 50 different objects across two devices (Redmi Note 10 and Vivo Y12):

• 43 out of 50 objects detected accurately (86% success rate)

• Camera quality didn't significantly affect results (all images resized to same dimensions)

• Main challenges: similar-shaped objects (vase/pot, glass/cup) and non-distinctive shapes (books, doors)

Black box testing — All 8 test procedures passed, such as: permission handling, camera detection, image capture and save, gallery navigation, delete functionality

User testing — 10 participants (4 students, 3 university students, 2 teachers, 1 general user) filled out a questionnaire:

• Average score: 88.8% — "Very Good" category

• Javanese language teachers validated 100% accuracy in script output

• Users reported the app made learning Javanese script feel natural and enjoyable

Future improvements to explore:

1. Try faster architectures (YOLO, EfficientDet) for smoother real-time performance

2. Train on a custom dataset with objects commonly found in Indonesia

3. Expand content beyond object names — phrases, numbers, sentences

But as a proof of concept, Njawani demonstrates something worth remembering: preserving culture doesn't mean going backward. Sometimes it means meeting people where they already are — through the screen of a smartphone.