Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/build/
/build-*/
/consumer-build/
/dist/
/install/
Expand Down
18 changes: 16 additions & 2 deletions ALGORITHM_PROVENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Algorithm provenance

Moonlight Audio Haptics 0.5 contains four provenance classes:
Moonlight Audio Haptics 0.7 contains five provenance classes:

1. the FFT, spectral/PCEN feature path, robust onset picker, IR mapping, and
platform-neutral engine are project-written Apache-2.0 code;
Expand All @@ -11,11 +11,25 @@ Moonlight Audio Haptics 0.5 contains four provenance classes:
3. `src/core/causal_rhythm_clock.*` is project-written Apache-2.0 code informed
by the published Real-Time Predominant Local Pulse method;
4. the GAME percussive/tonal features are project-written Apache-2.0 code
informed by published median-filter HPSS and SuperFlux methods.
informed by published median-filter HPSS and SuperFlux methods;
5. the ABI v2 authored-stereo analyzer is project-written Apache-2.0 code
using a first-order causal low-pass energy split and standard time-domain
amplitude, attack, zero-crossing, and cross-correlation measurements.

The SDK does not compile or link aubio. It has no third-party runtime binary or
model dependency.

## Authored-stereo fallback

`src/core/authored_haptics_engine.cpp` is a project-written streaming feature
extractor. Each source lane maintains its own state; no downmix, antiphase
fusion, pitch tracker, learned model, or device response curve is used. The
roughly 200 Hz split is a first-order exponential low-pass whose coefficient is
derived from the configured sample rate. RMS, peak, positive attack, zero
crossing rate, and normalized lane correlation use their standard definitions.
These features form a lossy transport fallback and are not claimed to recreate
the source PCM or estimate musical pitch.

## Public methods used

- The iterative radix-2 transform uses the standard Cooley-Tukey FFT
Expand Down
1 change: 1 addition & 0 deletions Android.mk
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := moonlight_haptics_core
LOCAL_SRC_FILES := src/core/audio_haptics_engine.cpp \
src/core/authored_haptics_engine.cpp \
src/core/causal_onset_detector.cpp \
src/core/causal_rhythm_clock.cpp \
src/core/feature_extractor.cpp \
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ while the public ABI has its own explicit version reported by

## Unreleased

- Add ABI v2 authored-stereo analysis for Sunshine fallback transports while
preserving the existing ABI v1 scene-authoring API.
- Preserve left/right source lanes in causal 5 ms IR frames with no processing
allocation, locking, device calibration, or actuator-specific curves.
- Reset cleanly on explicit discontinuity or sequence gaps and flush a marked
partial tail at end-of-stream.
- Add ABI-size, lane-isolation, chunk-invariance, discontinuity, timestamp, and
end-of-stream regression coverage.

## 0.6.0 - 2026-07-24

- Add a dialogue-aware GAME soft mask using the bundled WebRTC GMM VAD,
Expand Down
12 changes: 11 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
cmake_minimum_required(VERSION 3.16)

project(moonlight_haptics
VERSION 0.6.0
VERSION 0.7.0
DESCRIPTION "Portable real-time audio-to-haptics core"
LANGUAGES C CXX
)
Expand Down Expand Up @@ -54,6 +54,7 @@ set_target_properties(moonlight_haptics_fvad_objects PROPERTIES

add_library(moonlight_haptics_core STATIC
src/core/audio_haptics_engine.cpp
src/core/authored_haptics_engine.cpp
src/core/causal_onset_detector.cpp
src/core/causal_rhythm_clock.cpp
src/core/feature_extractor.cpp
Expand Down Expand Up @@ -163,6 +164,15 @@ if(MOONLIGHT_HAPTICS_BUILD_TESTS)
moonlight_haptics_enable_test_assertions(moonlight_haptics_abi_test)
add_test(NAME moonlight_haptics_abi COMMAND moonlight_haptics_abi_test)

add_executable(moonlight_haptics_authored_test
tests/authored_haptics_test.cpp)
target_link_libraries(moonlight_haptics_authored_test
PRIVATE moonlight_haptics_core)
moonlight_haptics_enable_test_assertions(
moonlight_haptics_authored_test)
add_test(NAME moonlight_haptics_authored
COMMAND moonlight_haptics_authored_test)

add_executable(moonlight_haptics_dsp_integration_test
tests/dsp_integration_test.cpp)
target_include_directories(moonlight_haptics_dsp_integration_test
Expand Down
25 changes: 23 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
Portable C++17 audio-to-haptics core shared by Android, HarmonyOS and future
server-side hosts such as Sunshine.

Version 0.5 keeps the stable ABI v1 and emits portable haptic IR from mono or
Version 0.7 keeps the stable scene-authoring ABI v1 and adds an additive ABI v2
Sunshine analysis path. The v1 engine emits portable haptic IR from mono or
multichannel PCM. It combines channel-aware spectral energy fusion, PCEN-style
adaptation, and robust onset picking with a portable Apache-2.0 adaptation of
AOSP HapticGenerator's causal actuator-shaped waveform. The AOSP waveform is
Expand Down Expand Up @@ -55,6 +56,21 @@ current ABI has no channel-layout mask, so the dialogue path uses a
layout-agnostic downmix and neutral centre evidence rather than guessing which
channel is centre. A future ABI may add an explicit channel layout.

The separate authored-stereo API in
`include/moonlight_haptics/authored_haptics.h` is for Sunshine deployments
where a client cannot receive original DualSense haptics PCM. It emits one
device-independent lane per source channel every 5 ms: RMS and peak amplitude,
attack strength, low-band energy ratio, and zero-crossing texture. The lanes
are never downmixed. Sequence gaps and explicit discontinuities reset filter
history and mark the first new frame so a client can ramp cleanly instead of
replaying stale energy. End-of-stream flushes a marked partial frame.

This authored IR is intentionally lossy and is not a replacement for raw PCM.
Sunshine should negotiate raw PCM first, analyze once only for clients needing
the fallback, and keep the analyzer outside the privileged capture sidecar.
Moonlight remains responsible for device calibration, actuator mapping, and
the final render curve.

## Build and test

```bash
Expand Down Expand Up @@ -91,7 +107,8 @@ human-readable release tag in a comment.

## ABI rules

- Public types live in `include/moonlight_haptics/audio_haptics.h` and compile as C11.
- Public types live in `include/moonlight_haptics/audio_haptics.h` and
`include/moonlight_haptics/authored_haptics.h` and compile as C11.
- Every public struct starts with `struct_size`; callers zero-initialize structs.
- `AhHapticFrame` is fixed at 80 bytes in ABI v1. Minor evolution consumes
reserved fields; changing the array element size requires a new API/ABI.
Expand All @@ -108,6 +125,10 @@ human-readable release tag in a comment.
32-bit atomics and may be updated from one control thread.
- A larger PCM input may span multiple analysis hops, so the caller provides a
fixed output array sized with `ah_get_max_output_frames()`.
- The ABI v2 authored-stereo analyzer accepts exactly two ordered haptics channels.
It preserves filter history across arbitrary input chunks, automatically
resets on a sequence gap, and exposes a separate capacity query that includes
a buffered partial window.

## License boundary

Expand Down
2 changes: 1 addition & 1 deletion VERSION.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.6.0
0.7.0
118 changes: 118 additions & 0 deletions include/moonlight_haptics/authored_haptics.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// SPDX-License-Identifier: Apache-2.0

#ifndef MOONLIGHT_HAPTICS_AUTHORED_HAPTICS_H
#define MOONLIGHT_HAPTICS_AUTHORED_HAPTICS_H

#include <stddef.h>
#include <stdint.h>

#include "moonlight_haptics/audio_haptics.h"

#ifdef __cplusplus
extern "C" {
#endif

/*
* Device-independent, two-lane authored haptics analysis.
*
* This API is a fallback for transports that cannot carry the original
* haptics PCM. It intentionally preserves the two source lanes and leaves
* actuator calibration and rendering to the client.
*/
typedef struct AhAuthoredEngine AhAuthoredEngine;

typedef uint32_t AhAuthoredInputFlags;
enum {
AH_AUTHORED_INPUT_NONE = 0,
AH_AUTHORED_INPUT_STREAM_START = 1u << 0,
AH_AUTHORED_INPUT_DISCONTINUITY = 1u << 1,
AH_AUTHORED_INPUT_STREAM_END = 1u << 2
};

typedef uint32_t AhAuthoredFrameFlags;
enum {
AH_AUTHORED_FRAME_NONE = 0,
AH_AUTHORED_FRAME_DISCONTINUITY = 1u << 0,
AH_AUTHORED_FRAME_PARTIAL = 1u << 1,
AH_AUTHORED_FRAME_STREAM_END = 1u << 2,
AH_AUTHORED_FRAME_SILENT = 1u << 3
};

typedef struct AhAuthoredConfig {
uint32_t struct_size;
uint32_t sample_rate;
uint32_t channel_count; /* Must be 2. */
uint32_t analysis_hop_frames; /* 0 selects 5 ms. */
uint32_t feature_flags; /* Must be 0 in ABI v2. */
uint32_t reserved[7];
} AhAuthoredConfig;

typedef struct AhAuthoredProcessInput {
uint32_t struct_size;
const int16_t* interleaved_pcm;
uint32_t frame_count;
uint32_t flags; /* AhAuthoredInputFlags bitset. */
uint64_t first_sample_time_us;
uint32_t sequence_number; /* Monotonic per stream, wraps naturally. */
uint32_t reserved;
} AhAuthoredProcessInput;
Comment thread
coderabbitai[bot] marked this conversation as resolved.

typedef struct AhAuthoredLaneFrame {
float rms_amplitude; /* Linear full scale, 0.0 .. 1.0. */
float peak_amplitude; /* Linear full scale, 0.0 .. 1.0. */
float transient_strength; /* Relative attack, 0.0 .. 1.0. */
float low_band_ratio; /* Energy below roughly 200 Hz, 0.0 .. 1.0. */
/* Sign transitions per second; a texture hint, not a pitch estimate. */
float zero_crossing_rate_hz;
uint32_t reserved[3];
} AhAuthoredLaneFrame;

typedef struct AhAuthoredHapticFrame {
uint32_t struct_size;
uint32_t flags; /* AhAuthoredFrameFlags bitset. */
uint64_t timestamp_us; /* End time of the represented PCM window. */
uint32_t source_sequence_number;
uint32_t source_frame_count;
AhAuthoredLaneFrame lanes[2]; /* Source order: left, right. */
float lane_correlation; /* -1.0 .. 1.0; diagnostic only. */
uint32_t reserved[3];
} AhAuthoredHapticFrame;

#define AH_AUTHORED_CONFIG_V2_SIZE 48u
#define AH_AUTHORED_PROCESS_INPUT_V2_SIZE \
((uint32_t)(offsetof(AhAuthoredProcessInput, reserved) + sizeof(uint32_t)))
#define AH_AUTHORED_LANE_FRAME_V2_SIZE 32u
#define AH_AUTHORED_HAPTIC_FRAME_V2_SIZE 104u

MOONLIGHT_HAPTICS_API AhStatus ah_authored_config_init(
AhAuthoredConfig* config,
uint32_t sample_rate);

MOONLIGHT_HAPTICS_API AhStatus ah_authored_create(
const AhAuthoredConfig* config,
AhAuthoredEngine** out_engine);

MOONLIGHT_HAPTICS_API uint32_t ah_authored_get_max_output_frames(
const AhAuthoredEngine* engine,
uint32_t input_frame_count,
uint32_t input_flags);

/*
* No allocation, locking, logging, or platform calls occur in this function.
* On AH_STATUS_BUFFER_TOO_SMALL, input is not consumed and out_count is zero.
*/
MOONLIGHT_HAPTICS_API AhStatus ah_authored_process_i16(
AhAuthoredEngine* engine,
const AhAuthoredProcessInput* input,
AhAuthoredHapticFrame* out_frames,
uint32_t out_capacity,
uint32_t* out_count);

MOONLIGHT_HAPTICS_API void ah_authored_reset(AhAuthoredEngine* engine);
MOONLIGHT_HAPTICS_API void ah_authored_destroy(AhAuthoredEngine* engine);

#ifdef __cplusplus
} // extern "C"
#endif

#endif // MOONLIGHT_HAPTICS_AUTHORED_HAPTICS_H
6 changes: 3 additions & 3 deletions include/moonlight_haptics/version.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
#define MOONLIGHT_HAPTICS_VERSION_H

#define MOONLIGHT_HAPTICS_VERSION_MAJOR 0
#define MOONLIGHT_HAPTICS_VERSION_MINOR 6
#define MOONLIGHT_HAPTICS_VERSION_MINOR 7
#define MOONLIGHT_HAPTICS_VERSION_PATCH 0
#define MOONLIGHT_HAPTICS_VERSION_STRING "0.6.0"
#define MOONLIGHT_HAPTICS_VERSION_STRING "0.7.0"
#define MOONLIGHT_HAPTICS_PARAMETER_SET_VERSION "action-rpg-p4g-v6-dialogue"

#define MOONLIGHT_HAPTICS_ABI_VERSION 1u
#define MOONLIGHT_HAPTICS_ABI_VERSION 2u

#endif // MOONLIGHT_HAPTICS_VERSION_H
2 changes: 1 addition & 1 deletion platform/android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ plugins {
}

group = 'com.moonlight.haptics'
version = (findProperty('sdkVersion') ?: '0.6.0-SNAPSHOT').toString()
version = (findProperty('sdkVersion') ?: '0.7.0-SNAPSHOT').toString()

android {
namespace 'com.moonlight.haptics.android'
Expand Down
Loading
Loading