Wearable IoT Audio Pendant Design & Development with Apollo510B & nRF54H20

Client: The client is a stealth mode, consumer-focused AI business producing a voice-enabled pendant with local artificial intelligence processing, secure MCU wearables and BLE connectivity.

Role: The team's role is the lead embedded systems developer (full stack by providing hardware integration, firmware, audio pipeline, security, and cloud back).

Outcome: The work effort is six months at full time equivalents. The team's output is a production-ready prototype (including a local wake word, a 4-mic beamforming system, encrypted audio buffers, and an end-to-end transcription pipeline).

This project involved developing a sophisticated voice-activated pendant from silicon to cloud.

The device captures high-quality audio from a 4-mic array, performs local wake word detection and command recognition using the Anker THUS AI chip, manages security and power with the Apollo510B, and handles BLE communication via the nRF54H20.

The pendant is provided with high-quality audio through a microphone array of four microphones for local/remote audio recording. The device utilizes an Anker THUS AI chip for detecting wake words and recognizing commands; this chip also manages security and power through Apollo510B. The device transmits data to a mobile application via nRF54H20 using BLE.

Through the use of an encrypted rolling buffer, all recorded audio is stored and is transmitted to an associated mobile application for Cloud Transcription Services.

The final version of the device is ready for mass production and provides users with the capability to retain their privacy (because it processes data locally), their security (because all data is encrypted during storage/transfer), and their ability for complex searches (using Cloud Transcription Services).

Processor Triad Overview

The features and capabilities available on this combination of three devices represent the intelligently-designed architecture for the secure master-controller-based wearable function of a pendant.

The Apollo510B serves as the primary, secure master controller by providing the critical functions of power management, encryption and orchestration for the complete operated system.

The ultra-low power architecture of the Apollo510B processor (30X more efficient than the Cortex-M4) is also critical for a pendant to provide one full day of use on a single small battery.

The nRF54H20 provides the ability to perform wireless communication independently due to its dedicated 48MHz wireless coprocessor, allowing the Apollo510B processor to enter a safe sleep state (while continuing BLE advertising) to conserve power.

This type of dual-processor architecture is essential for low-power wearables. The THUS Audio System (from Anker) offloads all audio AI processing from the nRF54H20 and provides secure location of sensitive voice data (in addition, provides a reduction in overall power consumption due to not streaming raw audio continuously).

Alternative considered: Using the nRF54H20 alone for both audio and BLE. Rejected because the Apollo510B's secureSPOT 3.0 provides hardware-isolated security domains essential for handling encrypted personal data.

Microphone Selection & Placement

The 4-mic array connects to the Apollo510B via its stereo digital microphone PDM interfaces. The Apollo510B supports up to 4 PDM channels simultaneously, making it ideal for this configuration.

Audio Processing Pipeline

(Pipeline diagram placeholder)

Implementation notes:

• The Apollo510B records raw PDM, then processes in PCM format. The audio is streamed to THUS from Anker via I2S at 16kHz for the wake word detection.
• When the wake word is detected, an interrupt is triggered in Apollo510B from Anker THUS.
• When Apollo510B receives the interrupt; it gets the last two seconds of the pre-roll buffer and continues to record until the end of the command.
• AES-256 encryption is used via the Apollo510B's hardware crypto accelerator (secureSPOT 3.0).

Anker THUS Integration

The Anker THUS chip is primarily a neural inference engine optimized for audio AI tasks. Configuration includes:

The training process was:

• Collected 500 wake word samples from 20 different speakers (varied accents and genders)
• Collected 2000 negative samples (ambient noise, other speech and other words)
• Trained using anker's machine learning (ML) training pipeline (details provided under NDA)
• Deployed as a 200 KB model file on THUS's internal memory directly.

The Apollo510B Cortex-M55 contains Helium (vector processing), which allows it to perform lightweight keyword spotting for a secondary (fallback) keyword trigger should the THUS be busy or in low power mode.

Architecture

The encrypted rolling buffer is a core privacy feature. Audio is never stored in plaintext — even on the device.

Implementation

The Apollo510B's secureSPOT 3.0 provides:

• Hardware-isolated key storage (inaccessible to firmware)
• True random number generator for IV/nonce generation
• Side-channel attack protection (timing/power analysis countermeasures)

Buffer operation:

• Audio is continuously captured (even before wake word)
• Each 256-byte chunk is encrypted with AES-256-GCM using a unique IV
• Encrypted data + auth tag (16 bytes) is stored in a circular buffer
• When buffer fills, oldest data is overwritten (rolling)
• On wake word, pre-roll (2 seconds before wake) + command (until silence) is extracted

Security benefit: If device is lost or stolen, audio cannot be recovered without the phone's private key (which never leaves the paired phone).

Key Exchange Protocol (BLE)

(Key exchange diagram placeholder)

Architecture

The nRF54H20 connects to the Apollo510B via SPI (high-speed, up to 32 MHz).

Advantages of using the Nordic nRF54H20 in the project:

• It features a dedicated 48MHz network processor, allowing for the BLE stack to run independently from the Apollo510B.
• It boasts a sensitivity rating of -100dBm when transmitting at 1Mbps, ensuring a reliable connection when the pendant is worn under clothing.
• The transmitter provides a maximum power output level of +10dBm, producing longer transmission distances and improved performance when worn as a body-worn device.
• It has earned PSA Level 3 certification, providing IoT users with the most secure method available, and matching the Apollo510B's security standards.

BLE Services Implemented

Performance Metrics:

• Audio Transfer Rate: 16 kB/s (16kHz/16-bit PCM)
• Achieved Throughput: ~25 kB/s (including overhead)
• BLE Connection Interval: 15 ms (low latency mode) and 200 ms (power save mode)
• nRF54H20 Tx Peak Current: ~4 mA
• Connected But No Data (Idle) Current: ~15 µA

Architecture

Pendant → BLE → Phone App → (optional) Cloud API → Transcription

Phone App Responsibilities (iOS & Android)

• Management of BLE Connections
• Decryption of Audio Chunks Using AES-256-GCM
• Reassembly of Command Files (audio format WAV) From Audio
• Optional Local Transcription Of Audio Chunk = On Device ASR
• Optional Cloud Transcription Of Audio Chunk = OpenAI Whisper, Google Speech-to-Text, or Custom

Cloud transcription flow (when used):

Phone → HTTPS/TLS → Cloud API → Transcription → Text returned to phone → Notification

The phone app handles all cloud interactions, not the pendant. This keeps the pendant off the internet entirely (privacy/compliance benefit).

Local vs Cloud Decision

System Power States

250 mAh Li-Polymer powered (small-sized pendant)

Battery Approximate runtime: approximately one day (full day equals approximately 12+ hours usage with typical usage of 50 commands/day)

Low power features supported by the Apollo510B

TurboSPOT technology supports "always-on" listening with Apollo510B.

• Subthreshold voltage operation that reduces leakage current
• 30 times better AI power efficiency than Cortex-M4
• By providing additional processing 16 times faster than if executing a wake word command as part of the Apollo510B's normal operation, back-up processing of wake words will complete faster.

Challenge 1: Synchronizing Audio between THUS and Apollo510B

Symptom: Each device has its independent audio capture path which causes them to lose their synchronization by a few samples over time.

Resolution: The PDM clock used by the Apollo510B is also used by the THUS (master clock of 12.288 MHz, divided down to generate 16kHz). An interrupt generated by the THUS on frame sync is used by the Apollo510B's rolling index of the buffer to keep the rolling indexes aligned to one another.

Challenge 2: Performance of Encrypted Buffers

Symptom: AES-256 encryption of each audio chunk introduced approximately 8 ms of latency per each audio chunk (256 bytes), which caused buffer overruns.

Resolution: The Apollo510B's hardware crypto engine (secureSPOT 3.0) can process an AES-256 encryption in less than 100 µs for each audio chunk. When switching AES from a software implementation to a hardware implementation, the encryption overhead was reduced by approximately 98%.

Challenge 3: Bluetooth Low Energy (BLE) Throughput with Encryption

Symptom: Due to the authentication tag (16 bytes) used by the AES-256 encryption of each audio chunk (256 bytes), that exceeds the BLE maximum transmission unit (MTU)(maximum of 256 bytes) causing audio chunks to be fragmented and sent over multiple packets.

Resolution: The BLE MTU was increased from 256 bytes to 512 bytes (this is supported by the Bluetooth 5.2 specification and nRF54H20 device). Therefore, each audio chunk fits into a single packet, reducing the overhead associated with multiple packets by ~40%.

The "dream version" audio pendant was successfully delivered as a fully functional production prototype. This prototype allows for:

• Privacy first architecture - capable of wake word detection on a local AI chip, sound never leaves the necklace in plaintext
• Security hardened - AES-256 encryption, secure keys stored in a hardware isolated manner (Apollo510B secureSPOT)
• Professional audio quality - has 4 microphones which form a beamforming array to attenuate background noise
• Seamless connectivity - nRF54H20 has a low power, robust Bluetooth low energy connection even when worn under clothing
• Scalable intelligence - local commands will provide low-latency capability and; cloud transcription will provide response capability to more complicated spoken requests

This has validated that the component triad of (Anker THUS + Apollo510B + nRF54H20 + 4-mic array) is not only a theoretical "dream" but also a practical and buildable system with established leading levels of security, power efficiency and audio quality.

Additionally, the AES-256 encrypted rolling buffer has also addressed the most significant consumer privacy issue with respect to always listening devices and, overall, the local-first architecture helps to minimize reliance on the cloud whilst preserving advanced features and benefits.